OSINT, short for Open-Source Intelligence, is 

data that has been obtained from

publicly available sources. OSINT is widely used within law enforcement work,
cybercrime activities such as planning an attack, or for business operation purposes,
such as checking out the competition. This course will primarily be focusing on the
application of OSINT tools and techniques for both red (offensive) and blue (defensive)
teams. Let’s explore some examples of OSINT data:

 A company that has a public web page that introduces some/all of their
employees (think a “meet the team!” page). An attacker could use this to very easily gather
a list of targets for social engineering attacks. Some sites may even include contact email
addresses or phone numbers, which can aid the social engineering process dramatically.

 Job Descriptions that leak information about a company’s internal systems (example:

a system administrator role requiring experience with Windows Server 2016 and Solaris).
An attacker can use this to plan internal attacks, lateral movement, and privilege escalation
once they have gained a foothold in the network.

 Photos on social media that are geotagged and contain device information in the
metadata(example: that nice photo you shared on holiday? we can find where that is,
usually in a matter of seconds – and the device you took it on).

 Reading a user’s social media profile to build up a profile of them (information such as

date of birth, locations, friends, interests, family). This can be used to learn more about an
individual, which is commonly done pre-interview so that employees can get a sense of
how the person will act in the workplace.

 Exposing Cyber Criminals. Whilst this begins to move into the Threat Intelligence domain,
it is possible to use OSINT sources and social-engineering skills to identify the true identity
of cybercriminals and pass the details to law enforcement. Whilst this is out of scope for
this entry-level course, it shows how powerful OSINT can be in the right hands.

 
As you can see, the above can be useful to both attackers and defenders. Attackers can
build up a good picture of their target without directly interacting with their systems,
whilst defenders can get a sense of the publicly available information that is present
on the internet, and work to reduce this or mitigate it using security controls such as
user awareness training and social-media policies.

 This activity is often referred to as conducting public exposure assessments.

Examples include gathering information employees put on social media, identifying
internet-facing assets, DNS checks, finding old login portals or websites, and much
more.

Profiling is the activity of collecting information on an individual to build up a picture of their

personality and behavior. This can be used to predict where they will be at certain times, based
on interests and previous locations. OSINT can be used to uncover the identities of
cybercriminals that have poor OPSEC (operational security – the practice of hiding yourself
online by disassociating your online persona with your real self). It can also be used to help find
missing persons (a great example of this is Trace Labs, a non-profit organization that hosts
online OSINT CTFs, which actually work to track missing people and assist law enforcement).

For Bussinesses
 

Businesses can utilize OSINT to keep an eye on the competition, watch for market activity, learn
more about their customer and how to best engage with them, improve business operations via
data enrichment, and also monitor for security risks such as leaked credentials, employees
sharing confidential information, or hackers planning attacks.

A big part of this is monitoring the social media channels of competitors to see what they're
doing well, as this can be used to bolster the company's social media and marketing strategies.

 
 

For Attackers
 

As we previously mentioned in the last lesson, OSINT sources can be a great way to discover
information about a target company or individual. By working out what systems a company uses,
the right exploits and attack methods can be planned out in advance. Employee information can
be harvested, allowing potentially effective social-engineering attacks, and spear-phishing email
campaigns to be conducted, tailored to their intended targets to make them more believable. A
company should be careful about what information its systems and employees are sharing online.
The process of collecting this information for malicious purposes is commonly referred to
as target information gathering, or passive information gathering (because the attacker is not
directly engaging with the target’s systems, such as port or vulnerability scanning).

Mark As Complete

Associated Roles
1. Introduction
2. An Introduction to OSINT
3. Associated Roles
 

There is a wide range of roles that will utilize OSINT in some capacity. Primarily Threat
Intelligence Analysts, Security Reseachers, and Penetration Testers will typically use OSINT
more than other roles in the industry.

 Tactical Threat Analyst

Tactical Threat Analysts may use OSINT to conduct intelligence operations, gathering
information on adversaries that may target their organization. By performing malicious actor
tracking, they can stay up to date with the latest trends and techniques used by these groups, in
order to implement defenses. They can also collect IOCs from OSINT sources, and use these to
conduct threat exposure checks internally.
 

 Strategic Threat Analyst

Strategic Threat Analysts may conduct threat exposure assessments, working to identify any
information that the organization is “leaking” out on the internet. This can be information
regarding internal systems, employees that are posting photos of their ID card on social media
(which can be used by malicious actors to copy it for social engineering attacks), and other cases
where information is present on the internet that could aid an adversary.

 Security Analyst

Security Analysts will use OSINT data for a number of reasons, such as checking the reputation
of IOCs such as IP addresses (VirusTotal, IPVoid), or email addresses and file hashes
(VirusTotal, IBM X-Force Exchange). Other use cases could include investigating fake social-
media accounts that are being used to launch social engineering attacks against employees.

 Vulnerability Analyst

OSINT is a crucial part of this role. Keeping up to date with the latest publicly-announced
vulnerabilities, following vulnerability researchers on social media, and information sharing is a
really important part of the work these guys do. A great OSINT source is the National
Vulnerability Database and using TweetDeck to monitor Twitter for vulnerability-related news
and disclosures.

 Penetration Tester/Red Teamer

Offensive security specialists will use OSINT to gain information about their target company,
such as internal systems and employee information. This will be used to tailor attacks, helping to
reduce the noise they make, and to make them more likely to succeed, resulting in a faster
penetration assessment.

A big part of this is monitoring the social media channels of competitors to see what
they're doing well, as this can be used to bolster the company's social media and
marketing strategies.

 
The process of collecting this information for malicious purposes is commonly
referred to as target information gathering, or passive information
gathering (because the attacker is not directly engaging with the target’s systems,
such as port or vulnerability scanning

 Strategic Threat Analyst

Strategic Threat Analysts may conduct threat exposure assessments, working to

identify any information that the organization is “leaking” out on the internet. This can
be information regarding internal systems, employees that are posting photos of their
ID card on social media (which can be used by malicious actors to copy it for social
engineering attacks), and other cases where information is present on the internet that
could aid an adversary.

 Security Analyst

Security Analysts will use OSINT data for a number of reasons, such as checking the
reputation of IOCs such as IP addresses (VirusTotal, IPVoid), or email addresses and file
hashes (VirusTotal, IBM X-Force Exchange). Other use cases could include investigating
fake social-media accounts that are being used to launch social engineering attacks
against employees.

 Vulnerability Analyst

OSINT is a crucial part of this role. Keeping up to date with the latest publicly-
announced vulnerabilities, following vulnerability researchers on social media, and
information sharing is a really important part of the work these guys do. A great OSINT
source is the National Vulnerability Database and using TweetDeck to monitor Twitter
for vulnerability-related news and disclosures.
 

1) Planning and Direction

 

This first stage is the crucial element of the research process, it is the moment when you define
which horizon will your investigation take.

This is where you determine the purpose of your research and what kind of information you are
looking for.

2) Collection (gathering of data and information)

 

In this second phase your objective will be the identification of which kind of processes you will
use to carry out the collection of such information, and then, using all the techniques you know,
obtain the data that will help you carry out your intelligence operation.

 
3) Processing of data and information
 

In this phase, you will take care of all that was obtained in the previous process.

Here your objective is not only the visualization of the information but also the application of
decoding, decryption, validation, and evaluation techniques that will allow you to filter the huge
amount of information you obtained, to identify useful data for your research.

4) Analysis to produce meaningful intelligence

 

This is where analysts can show what they really are.

Here you must compile all the information you filtered in the previous step to obtain the solutions
to your initial problem, as well as the creation of a coherent intelligence product (report,
conference, etc.) that allows you to clearly explain the process you recently carried out

 
5) Dissemination of intelligence to the clients
 

And finally, we have the final step. Here you must deliver the product you developed throughout
the process to the stakeholders (individuals or groups) that requested it. This will help these
people make informed and appropriate decisions when tackling the original problem.

That is all about the intelligence cycle, now you know everything you have to know to carry out an
intelligence process and succeed in the process.

 Materials
If you want to know more about this topic you should check the following links:

 https://www.intelligencecareers.gov/icintelligence.html
 https://www.sciencedirect.com/topics/computer-science/intelligence-cycle
 https://www.e-education.psu.edu/sgam/node/15
 https://www.groupsense.io/resources/how-to-use-the-intelligence-cycle-to-secure-your-
brand

On the following websites, you can learn more about the different cookie varieties and
how they allow websites to track you:

 https://www.theguardian.com/technology/2012/apr/23/cookies-and-web-tracking-intro
 https://privacy.net/stop-cookies-tracking/
  https://www.cookieyes.com/how-cookies-track-you-on-the-web-and-what-to-do-about-it/


 https://coveryourtracks.eff.org
  


https://browserleaks.com
 It would also be a good idea to review the following Checklist provided by
Inteltechniques.com: https://inteltechniques.com/JE/Privacy_Checklists_Feb2019.
pdf
  

Below, you will find some of the best guides on the Internet to build your own Sock
Puppet and the best advice on how to NOT build one.

 “Welcome to the (Sock) Jungle”: https://youtu.be/v8EP6xOcB8M

 “The Art of The Sock”: https://www.secjuice.com/the-art-of-the-sock-osint-humint/
 “Creating an Effective Sock Puppet for OSINT
Investigations”: https://web.archive.org/web/20210125191016/https://jakecreps.com/
2018/11/02/sock-puppets/
 “Sock puppet Account Creation – My Process”: https://garrettmickley.com/sockpuppet-
account-creation/
 “The World’s Best Sock Puppet…Not!”: https://keyfindings.blog/2019/01/21/the-worlds-
best-sock-puppet-not/

The Harvester is a command-line information-gathering tool that utilizes OSINT sources to

gather information about the target domain and retrieves information such as hostnames, IP
addresses, employees (and their positions), email addresses, and much more. In the below
screenshot, I am performing simple reconnaissance on the domain Google.com, using Google as
the data source:

theharvester -d google.com -l 100 -b google

(tool) (target domain = google.com) (list 100 results max) (source = google)

 
 

This didn’t give us too much information, but knowing the IPs associated with google
subdomains could be useful. Now let’s try something a little bit different. If we wanted to launch
social engineering attacks against some Google employees, we can quickly identify potential
targets by setting the data source to be ‘Linkedin’ instead of ‘Google’:

theharvester -d google.com -l 100 -b linkedin

(tool) (target domain = google.com) (list 100 results max) (source = linkedin)

 
 

Now we have a list of potential targets, along with their job titles. We can do further
reconnaissance on them using Linkedin itself and build up a profile on them using a tool like
Maltego, then we can launch spear-phishing attacks as part of a threat simulation engagement!

Try using The Harvester with different domains, and different data sources. You can access the
help sheet for this tool by using the command theharvester within the command line. This will
show you the data sources that are available, and other arguments to retrieve specific data.

Mark As Complete

Maltego
1. Tools and Services
2. Tools and Services
 3. Maltego

Following the line of TheHarvester, we will now have a very powerful OSINT tool called
Maltego.

Maltego is a high-level data mining and information gathering tool, capable of obtaining real-
time data on different types of entities (companies, people, websites, etc.), and representing them
graphically through nodes, showing all the connections that the program was able to obtain over
the Internet, about the subject under investigation.

Note: For this explanation, we will make use of the Kali Linux Operating System, since this tool
is part of its “Information Gathering” collection.

So, first things first, to use Maltego simply type the command “maltego” on the command line,
or search for the application through the Kali toolbar, as shown in the gif below.
 

Once started, you will see that there are 5 different versions of the software. Some are paid
versions, while others are not. This time we will use the Community Edition since it is free and
should not be a problem for you to use it.
 

Once you have selected your version, all you have to do is accept the license terms, create an
account (you will then receive an API key that will allow you to use Maltego), and select the
settings you prefer.

Once all the above is done, select the option “Open a blank graph” and finish the process. This
will allow you to visualize a screen similar to the one presented below:

 
 

The next thing we should do to perform our first search would be to go to the “Transforms” tab
and “Transform Hub”.

This will display the catalog of Transform Tools that can be installed in Maltego
 

For this example, install the tools “CaseFile Entities”, “HaveIBeenPwned?”, “Social Links
CE” and “Shodan” (In the case of Shodan you must enter an API Key. To do this simply create
an account on their website www.shodan.io and go to your account)

Once you have installed all the Transfer Tools, go back to the blank tab we created earlier and in
the “Entity Palette” select “Domain” (you can use the search bar located in the same section),
and drag it to the screen, as you can see in the following gif:
 

(In this example, we will make a scan of the site “paterva.com”, since they are the creators of
Maltego and have given their permission to carry out this kind of scan).
Once you’re done with it, right-click on the entity, and click on the arrow next to “All
transforms”, this will start the scan.

(If this is your first time running Maltego, a window will be displayed asking you to accept the
disclaimer)

 
 

Once the scan is complete, you should obtain a graph similar to the one below, displaying all the
information obtained by Maltego during the scan.
 

And that’s it, you’re ready to use Maltego, now start exploring all its functions and analyze all
the information you got. The rest is up to you!

If you want to learn more about how to use this software, click on the following link and check
out all the resources available on Maltego’s official
site: https://www.maltego.com/categories/tutorial/

Following the line of TheHarvester, we will now have a very powerful OSINT tool called
Maltego.

Maltego is a high-level data mining and information gathering tool, capable of obtaining real-
time data on different types of entities (companies, people, websites, etc.), and representing them
graphically through nodes, showing all the connections that the program was able to obtain over
the Internet, about the subject under investigation.

Note: For this explanation, we will make use of the Kali Linux Operating System, since this tool
is part of its “Information Gathering” collection.

So, first things first, to use Maltego simply type the command “maltego” on the command line,
or search for the application through the Kali toolbar, as shown in the gif below.
 

Once started, you will see that there are 5 different versions of the software. Some are paid
versions, while others are not. This time we will use the Community Edition since it is free and
should not be a problem for you to use it.
 

Once you have selected your version, all you have to do is accept the license terms, create an
account (you will then receive an API key that will allow you to use Maltego), and select the
settings you prefer.

Once all the above is done, select the option “Open a blank graph” and finish the process. This
will allow you to visualize a screen similar to the one presented below:

 
 

The next thing we should do to perform our first search would be to go to the “Transforms” tab
and “Transform Hub”.

This will display the catalog of Transform Tools that can be installed in Maltego
 

For this example, install the tools “CaseFile Entities”, “HaveIBeenPwned?”, “Social Links
CE” and “Shodan” (In the case of Shodan you must enter an API Key. To do this simply create
an account on their website www.shodan.io and go to your account)

Once you have installed all the Transfer Tools, go back to the blank tab we created earlier and in
the “Entity Palette” select “Domain” (you can use the search bar located in the same section),
and drag it to the screen, as you can see in the following gif:
 

(In this example, we will make a scan of the site “paterva.com”, since they are the creators of
Maltego and have given their permission to carry out this kind of scan).
Once you’re done with it, right-click on the entity, and click on the arrow next to “All
transforms”, this will start the scan.

(If this is your first time running Maltego, a window will be displayed asking you to accept the
disclaimer)

 
 

Once the scan is complete, you should obtain a graph similar to the one below, displaying all the
information obtained by Maltego during the scan.
 

And that’s it, you’re ready to use Maltego, now start exploring all its functions and analyze all
the information you got. The rest is up to you!

If you want to learn more about how to use this software, click on the following link and check
out all the resources available on Maltego’s official
site: https://www.maltego.com/categories/tutorial/

Google is helpful in general, but Google Dorks are search hacks where we can use
special arguments in a normal Google query to find specific information. Dorks come
in the format operator:keyword, an example of this would be filetype:pdf. Real-world
examples of using Dorks include:

 Retrieving files from domains.

 Finding hidden web pages and login portals.
 Subdomain enumeration.
 And more!

 
 

Finding Files
 

Let’s start of by seeing what PDFs we can find that are associated with cyber security,
using the dork query:
Cyber Security filetype:pdf .

In the above screenshot we can see that the search has in fact brought back any PDF
files that contain the strings “cyber” and “security. We can use this to see what files a
company is hosting online, and see if any are confidential and should not be publicly
accessible. It is also possible to retrieve information about internal systems and users
by looking at the metadata of files to see when they were created and who by.
Documents can also be used to create custom wordlists for password attacks against
specific organizations. Try this yourself with a search term or a domain (such as
Facebook.com filetype:pdf).

Subdomain Enumeration
 

Now let’s see how Dorks can be used to enumerate all subdomains of a domain, for
passive reconnaissance purposes. For this, we will use Facebook again as the example,
with the following query:
site:Facebook.com -site:www.Facebook.com
(Look for sites that include .Facebook.com) (but NOT www.Facebook.com)

 
 

Here we can see the list begins with two subdomains, portal.facebook.com, and
code.facebook.com. We have successfully enumerated subdomains using Google
Dorks. This is a great method for identifying uncommon web pages that may feature a
login portal or valuable information such as development environments, files, and
more. Have a go yourself with any domain you want, and see what interesting
subdomains they have!

Using the Dork inurl: (value) we can look for specific keywords in a much more refined
way than normal google searches. Using the query inurl: admin we can see a number
of what appears to be admin login portals. This would be great if we were working as
an attacker (if in scope, we can brute force or bypass the login portal to access
administrator dashboards) or a defender (we can work to secure these portals so they
are not compromised).

Conclusion
 

As you now know, Google Dorks are very useful. Take a look at this extensive list of the
different search queries you can use, and play around with a few yourself to really
understand how they work, and how they could be used for defensive or offensive
security. https://securitytrails.com/blog/google-hacking-techniques

 
We know that Google Dorks can be incredibly powerful – they can exposure admin
login portals, usernames and passwords, IP cameras and webcams, and much more.
But how exactly do we protect against them?

Geofencing and IP Whitelisting

 

There are a number of ways we can use IP-based controls to restrict who can access
web content, such as unauthorized users or google’s crawlers (programs that search
the internet, indexing every publicly-accessible page they can find).

Geofencing is a method of blocking entire IP ranges associated with countries so that

only access from a specific location will be allowed. For example, if a UK business
wanted to only allow connections to their site from UK IP addresses, they could do
this. Can you see the security flaw in this? If someone from America had a VPN, they
could set it to exit from the UK, and now they are able to access the site. Netflix uses IP
geofencing as they have different versions of the USA and the UK. VPNs were used to
access the American version, but soon after Netflix caught on, and started to disallow
VPN connections and proxies to access Netflix (you’ll see an error page asking you to
turn off your routed connection before being able to access content).

IP whitelisting works in a similar way, but instead it only allows specified IP addresses
to access resources, and blocks everything else. This is great if you have a
development environment or site that is present on the internet, but you don’t want
anyone accessing it. You can set the whitelisted IPs to the public range of the
organization so that only IPs belonging to the company can actually view the site.

 
Crawler Restrictions
 

A very effective method of preventing content being present on Google is to block

their crawlers from being able to access any of the content present on the internet.
This can be achieved by creating a robots.txt file that disallows any crawlers from
indexing any part of your website, preventing it from showing on Google’s search
engine.

The above file will disallow any (*) user-agents from crawling any directory on the
website (such as /images/ or /training/)

Requesting Content Removal

 

If you run google dorks against your company and find results that could be used by a
malicious actor (such as a login portal or sensitive files) you can make a request to
Google asking them to temporarily remove the content from their search engine (90
days) or permanently remove it. You must provide sufficient evidence that you own
the site in order to have anything removed.
Information about temporary and permanent URL removals can be requested here –
https://support.google.com/webmasters/answer/1663419?hl=en

Without diving down the rabbit-hole that is Threat Intelligence, we can quickly get a sense of
whether a target email address has been mentioned in a data breach. Why is this useful? Because
if we get an indication it has been leaked before, we can then start to explore paths such as
finding data breach dumps on the dark web, and seeing if the email address has been linked with
any passwords – then we can use these for password or social-engineering attacks. If we visit
the Email Address branch, then the Data Breach sub-branch, we are provided with a number of
online services that allow us to enter email addresses in, to see if they have been breached.

Use Case: Boosting OSINT Skills And

Knowledge
 
OSINT Framework offers a good selection of OSINT training resources, so if you’re looking to
further your skills then check these out. These are available under the Training branch.

We strongly suggest you check out this tool and see what interesting sites and tools you can find
from it. https://www.osintframework.com