GUIDE

Introduction To Open-Source
Intelligence Tools
Spring 2020

www.datawalk.com

Copyright © 2020 DataWalk. All rights reserved.

 Introduction
“Successful investigations require fast, efficient, and
All over the world, agencies incorporate publicly last few months we also have been observing how adaptable methods to combine data from different sources
accessible data from the Internet - Open Source EU-based GDPR and other US-based regulations
Intelligence, or “OSINT” - in their investigations. affect OSINT capabilities, even though law
and the use of intuitive visualization to efficiently spot
Through a number of techniques, it’s possible to use enforcement use has a different legal framework for patterns and trends.”
OSINT to uncover traces unwittingly left by dealing with personal data. Further, one of the most
criminals. Currently available tools make it possible common OSINT services run by Michael Bazzell shut Chris Westphal - Chief Analytics Officer, DataWalk
to identify the true identities of Internet users, and down the free tool section of his website
define their patterns of life or connections with inteltechniques.com, including the Facebook
other users. However, what is crucial here is to investigative tools that were powered by the Graph
quickly combine collected OSINT data with your Search feature. This section still exists for paid "OSINT typically enables profiling people and companies to
other data, putting it in context as well as verifying subscribers to Bazzell’s services. Similarly, the
circumstantial evidence. popular people search service pipl.com announced
understand the real motives, but sometimes it only gives an
that it would close its free service, though a paid outline of certain possibilities or hypotheses."
This document is a brief guide to present selected, premium service is still available.
practical OSINT tools that will help you in your Jakub Karmowski - OSINT Expert
everyday work. However, it is necessary to bear in In addition to OSINT tools for the world wide web, in
mind that the world of OSINT tools is constantly this guide we’ll also discuss the “Dark Net” where
changing, and the tools that work effectively in a criminals trade in illegal goods and services. At the
particular area today may turn out to be of no use end of this guide you will find a two-page cheat-
tomorrow, as a few examples show below. sheet which you can print and then hang on the wall
as a handy reference. Please note that each tool
Recently there have been many changes in the listed includes a link to its website.
OSINT world. Social media services have made the
discovery of information increasingly difficult, with
June 2019 being an exceptionally challenging month DataWalk Team
because of new restrictions on Facebook Graph. [email protected]
Facebook discontinued the Graph Search capability.
Even though it wasn’t removed entirely it limited
access to Facebook data by ordinary users. In the

www.datawalk.com 1/37
 Multi-Search Engines

www.datawalk.com 2/37
 Advanced Search Engines

Use free tools below to query multiple search engines simultaneously Highly recommended The most classic of advanced search engines. Allows you to
search Google resources in an advanced way without using
google.com/ separate queries. Also use specific forms i.e. site:.gov
advanced_search filetype:csv or “” to get precise results.
Highly recommended

intelx.io/tools Search Highly recommended Yandex Search is a web search engine. Yandex Search also
features “parallel” search that presents results from both main
yandex.com web index and specialized information resources, including
Highly recommended news, shopping, blogs, images and videos on a single page.

osintframework.com Search Baidu offers various services, including a Chinese search

engine, as well as a mapping service called Baidu Maps. Baidu
baidu.com offers about 57 search and community services, such as Baidu
Baike (an online encyclopedia) and a keyword-based discussion
Highly recommended
forum.

cyber-cops.com/investigation-tools Search Highly recommended

Bing is a web search engine owned and operated by Microsoft.
bing.com Bing provides a variety of search services, including web, video,
image and map search products.

Qwant is a European web search engine. The website processes

qwant.com well over 10 million search requests per day and over 50 million
individual users a month worldwide.

Highly recommended
Search engine for websites available on the Tor network, for
ahmia.fi sources that are commonly known as DeepWeb or darknet.
Ahmia provides hidden content not only to Tor users.

www.datawalk.com 3/37
 Metasearch Engines

www.datawalk.com 4/37
 Metasearch Engines

A Metasearch engines are search tools that use another Highly recommended Highly recommended
search engine's data to generate its own results. Metasearch
engines simultaneously send out queries to third party alltheinternet.com myallsearch.com
search engines and cumulate and summary results.

Highly recommended Highly recommended Highly recommended Highly recommended

all-io.net/ zapmeta.com searchall.net instya.com

Highly recommended Highly recommended Highly recommended

duckduckgo.com webcrawler.com infospace.com

Highly recommended Highly recommended

search.carrot2.org etools.ch

Note that in this guide we present only a sample of available tools. The
success of your searches depends on defining the purpose of your
search, the choice of tool, and your imagination!

www.datawalk.com 5/37
 Data Breach / Leak

www.datawalk.com 6/37
 Data Breach / Leak

Have I Been Pwned allows you to search across multiple data breaches to
see if your email address has been compromised

Highly recommended
haveibeenpwned.com

Is your customer data among billions of compromised data assets?

Highly recommended
dehashed.com

Pastebin dump collection - get dumps by date, domain, search by email, etc.

Highly recommended
psbdmp.ws

Specific search terms such as email addresses, domains, URLs, IPs, CIDRs, Bitcoin
addresses, IPFS hashes, etc

Highly recommended

intelx.io

www.datawalk.com 7/37
 Social Media Search Engines
Various tools are available to help you find information on particular people, users,
ID numbers, trends, keywords, mutual acquaintances, visited places, and “likes” on
Facebook, Twitter, Instagram, and other social networks.

www.datawalk.com
 Social Media Search Engines
Highly recommended Highly recommended Highly recommended Highly recommended
social-searcher.com wiki.kenburbary.com/social- shadowdragon.io hashatit.com
meda-monitoring-wiki
Free browser of social media. Without ShadowDragon is a cybersecurity Everywhere on social media, content is
logging in, users can search information A set of free and paid tools to monitor solutions developer that simplifies being generated at unheard of speeds.
published on Twitter, Google+, social media content and user complex multi-technology, multi- Hashtags help you navigate the ever-
Facebook, YouTube, Instagram, LinkedIn, activities. environment digital investigations with expanding internet, and HASHATIT keeps
Tumblr, Reddit, Flickr, Dailymotion and easy-to-use tools. you on top of hashtags.
Vimeo.

Highly recommended Highly recommended Highly recommended Highly recommended

twitterfall.com socialbearing.com twitonomy.com followerwonk.com

Twitterfall is a web-based Twitter client
where tweets fall down the screen like a Twitter analytics. You can use this to Browse, search, filter and get insights Followerwonk helps you explore and
waterfall. Twitterfall takes advantage of find, filter and sort tweets or search/ on the people you follow and those who grow your social graph. Dig deeper into
Twitter's search trends, which reveal filter by influence, engagement, follow you. Backup/export tweets, Twitter analytics: Who are your
the topics that are currently most sentiment, location and more. Also retweets, mentions and reports to Excel followers; where are they located and so
popular and most discussed at that statistics for accounts are available. & PDF. forth.
time.

Highly recommended Highly recommended Highly recommended Highly recommended

tweetbeaver.com omnisci.com unionmetrics.com hashtagify.me
TweetBeaver gathers the data you need OmniSci is a GPU (Graphics Processor Free Twitter analytics. Quick answers to Hashtagify is one of the most advanced
quickly, simply and easily. This works at Unit)-powered database and your biggest Twitter questions with our Twitter hashtag tracking tools. It allows
a granular level, digging out the visualization platform designed for free snapshot report. Analyze recent you to find the best hashtags to reach
information you need from your or any lightning-fast, immersive data conversation about anything on Twitter your audience, gives you custom
other public account. It can gather data exploration that eliminates the – your account, your competitors, a suggestions, and helps you get to know
on any non-private account and returns disconnect between analyst and data. campaign, a hashtag, a trend, or a your users better.
most searches as a csv. Tweet.

Highly recommended Highly recommended Highly recommended

spoonbill.io feed.exileed.com talkwalker.com

Spoonbill lets you see profile changes

from the people you follow on Twitter.
Keep updated on your friends' and RSS for Twitter. The real-time FREE social media search
family members' bios, websites, engine.
locations, and names.

www.datawalk.com 9/37
 Search Engines
Of People And Business
Email Addresses
Looking for information on particular people, email addresses or telephone
numbers? Want to check an email address of a person of interest on the basis
of a domain? These search engines will help you identify hidden content.

www.datawalk.com
 Search Engines Of People And Business Email Addresses

Highly recommended

spokeo.com webmii.com skymem.info

Highly recommended
A search engine of people and a catalog of WebMii is a next-generation search engine: E-mail address search engine -
pipl.com telephone numbers, email addresses and find all information on people and get an searches 150 million records in a
photos. Look for people by their surnames, evaluation of their visibility in the web. few seconds.
Pipl's unique identity resolution email addresses, addresses and phones
engine connects the world's personal, for free.
professional and social identity data to
give analysts and investigators an
unmatched global index of over 3 billion
trusted identity profiles.
Highly recommended Highly recommended Highly recommended

connect.clearbit.com hunter.io voilanorbert.com

Find any email in under five seconds. A solution for finding and verifying A comprehensive search engine for people.
professional email addresses. You can find a person on the basis of their
email address.

www.datawalk.com 11/37
Search Engines Of Domains,
Websites, Emails And IP
Registry
Looking for information on particular domains or IP addresses?
Take advantage of search engines for exploration and investigation.

12/37
 Search Engines Of Domains, Websites, Emails And IP Registry

Highly recommended Highly recommended Highly recommended

hexillion.com whois.net website.informer.com

Get a quick aggregated view of

Tools for investigating, exploring, everything the Web can promptly tell
and troubleshooting Internet addresses Whois Lookup — Domain Names Search, you about a site (domain), including its These are tools that search
such as domain names, IP addresses, Registration and Availability. daily visitors, safety status, Alexa rank,
email addresses, and URLs. owners and much more. the web for domain names, IP
addresses, email addresses,
and URLs. These tools
Highly recommended Highly recommended Highly recommended process and combine
myip.ms viewdns.info builtwith.com information from various
public sources, such as
Whois, DNS records and web
BuiltWith technology tracking includes
widgets, analytics, frameworks, content pages.
Find Hosting Company Of Any Website or management systems, advertisers,
owner of any IP address. Check DNS info of websites. content delivery networks, web
standards and web servers to name
some of the technology categories we
cover.

Highly recommended Highly recommended To visualize complexity of domains use:

urlscan.io censys.io
http://www.visualsitemapper.com/
urlscan.io is a service to scan and
analyse websites. When a URL is With full visibility over your entire attack To search for similar websites use:
submitted to urlscan.io, an automated surface in near real time, you can
process will browse to the URL like a regularly monitor and mitigate risks to
regular user and record the activity that prevent unnecessary exposures. http://analyzeid.com/
this page navigation creates.

www.datawalk.com 13/37
 Search Engines Of Domains, Websites, Emails And IP Registry

Highly recommended Highly recommended Highly recommended

analyzeid.com securitytrails.com spyonweb.com

Spend less time researching website SpyOnWeb.com takes the information

owners with our reverse look up tool. See through the external internet from public sources, then structures it
Find all their websites that linked surface area of any company from a for your quick and convenient search for
to Adsense, Google Analytics, Amazon single unified web interface. the websites that probably belong to
Affiliate, Email, etc. the same owner.

Highly recommended Highly recommended Highly recommended

virustotal.com zoomeye.org visualsitemapper.com

Analyze suspicious files and URLs to Cyberspace Search Engine recording

detect types of malware, and Visual Site Mapper is a free service that
automatically share them with the information of devices, websites, can quickly show a map of your site.
security community. services and components, etc.

www.datawalk.com 14/37
Deep Reconnaissance

15/37
 Deep reconnaissance

Highly recommended

spiderfoot.net

SpiderFoot is a reconnaissance tool that automatically

queries over 100 public data sources (OSINT) to gather
intelligence on IP addresses, domain names, e-mail
addresses, names and more.

Highly recommended

spiderfoot.net/hx/

Automatically scan targets according to a daily, weekly or

monthly schedule and get notified when changes in their
OSINT footprint are detected.

www.datawalk.com 17 /20
16/37
 Other OSINT Tools

www.datawalk.com 17/37
 Other OSINT Tools

This test will display a list of MX records (IP addresses) for a

domain according to priority. You can click Diagnosis to connect to
mxtoolbox.com a mail server, verify reverse DNS records, perform a simple Open
Relay test and check the response time. Also, you can check each
MX record against 105 blacklists based on DNS.

Highly recommended
Enter a user name and click one button to check if the name is
already in use, across dozens of social networking sites. In
namechk.com addition, check the availability of domains of this name. Forget the
painstaking task of visiting each site to check; with namechk you
can easily get a condensed result for free.

Highly recommended
Most tools free Internet archive, allows you to search, among others archival
versions of web pages (wayback machine). Thanks to this you
archive.org have access to unique archival information, e.g., bank accounts,
phone numbers, e-mail addresses, and also images and other
content that can provide context of the case under investigation.

Highly recommended
Most tools free This tool is designed to provide a quick method to search
usernames and hashtags on TikTok via a browser. All results take
osintcombine.com/tiktok- the user to the source on the TikTok.com website or are provided
quick-search as a Google search result. NO data is stored, managed or
processed by OSINT Combine.

www.datawalk.com 18/37
 Other OSINT Tools

Want to find similar websites? Just enter a domain name and you
will immediately get information about services with similar
content. You can check websites visited by other users and
similarsites.com keywords they were looking for. Also, you will find out if a website
of interest is active and how much time users spend there on
average.

Highly recommended

Debouncer checks for the occurrence of IP addresses and mail

debouncer.com server domains on blacklists, based on DNS (DNSBL lub RBL).

Highly recommended
Takes a domain or IP address and does a reverse lookup to quickly
show all other domains hosted from the same server. Useful for
viewdns.info/reverseip finding phishing sites or identifying other sites on the same
shared hosting server.

Highly recommended

NameCheckup is a search tool that allows users to check social

namecheckup.com media username availability over many social networks and sites,
and to check domain availability at the same time.

www.datawalk.com 19/37
 Other OSINT Tools

Highly recommended

Hunchly automatically collects, documents, and annotates every

hunch.ly web page you visit.

Highly recommended

osintcombine.com/google- This tool is designed to assist in tracking associated webpages

analytics-id-explorer which share the same Google Analytics ID.

www.datawalk.com 21/30
 Web Scrapers
If you want to download large amounts of
data from particular web sites for further
analysis, here are some tools that can help.

www.datawalk.com 21/37
 WEB SCRAPERS

Highly recommended

When used as an extension to a browser (e.g., Chrome), Webscraper.io enables you to create a map
webscraper.io of a website so that you can decide which data should be extracted. Scanned data can be exported
to a CSV file.

octoparse.com data-miner.io

Octoparse is a powerful web scraper which is easy to use. It can work with A very easy to use Chrome extension that enables you to scan data from
both static and dynamic websites by means of AJAX, JavaScript, cookies, tables and lists on separate pages to CSV or XLS files.
etc. You can download an installer and run a data download task on any
website, even if the website is paginated or requires logging-in.

Only MS Windows version

www.datawalk.com 22/37
 Cryptocurrency
Analysis And Lookup

www.datawalk.com 23/37
 Cryptocurrency analysis and lookup

Highly recommended Highly recommended Highly recommended

xmrhunter.com blockchain.com/explorer walletexplorer.com

This site is intended for Security A real-time search engine of Bitcoin,

Analysts who want to gain quick insight Ethereum, and Bitcoin Cash. You can Bitcoin block explorer with address
into the scale of a Mining operation search for wallets addresses, grouping and wallet labeling.
(Monero). transaction and blocks.

Highly recommended Highly recommended

moneroblocks.info bitcoinwhoswho.com

Monero blockchain explorer - XMR Check a bitcoin address to see if there

blocks, transactions, payment ids, have been scam alerts connected to it.
hashrate, emission. They show it all. Or report a scam if you have details on
one.

www.datawalk.com 24/37
 FTP Search

www.datawalk.com 25/37
 FTP Search

Highly recommended
globalfilesearch.com

Global File Search is the search engine

allowing you to find a file among
millions of files located on FTP servers.
The search engine database contains
regularly updated information on the
contents of thousands FTP servers
worldwide. FTP Search engine does not
search the contents of the files.

FTP

Highly recommended

searchftps.net

This website is intended to provide a

very powerful FTP indexing and search
service allowing you to find a file among
millions of files located on FTP servers.

www.datawalk.com 27/30
26/37
 Maps
Various advanced maps, satellite imagery and information.

www.datawalk.com 27/37
 Maps

Highly recommended
search.earthdata.nasa.gov/search

Highly recommended

Highly recommended dgi.inpe.br/CDSR/

earthexplorer.usgs.gov

Highly recommended

digitalglobe.com/ecosystem/open-data
Highly recommended

worldview.earthdata.nasa.gov

www.datawalk.com 28/37
 The Deep Web
And The Dark Web
Most people have access to only the indexed
World Wide Web (www) which is only perhaps 6%
of all content published on the Internet. The most
popular search engine for this is Google.

So what is beyond the indexed horizon?

www.datawalk.com 29/37
 World Wide Web
The Deep Web And The Dark Web
Only about from 1% to 6% of the
content on the Internet is openly
available and indexed by search
Deep Web engines such as Google.
The deep web is part of the World Wide Web whose contents are not indexed by standard web
search engines. There are many common examples of deep web sources, such as private content
on social media sites like Facebook and Twitter, as well as email messages, chat messages,
electronic bank statements, electronic health records (EHRs), etc. There are tools (e.g.,
IntelTechniques) that allow access to some of this information.

Highly recommended
Hidden Wiki Deep Web links Deep Web
hiddenwikitor.com
Over 90% of the
information on the
Highly recommended Internet is in the deep

ahmia.fi Ahmia searches hidden services on the Tor network 7.9 web (e.g., password
protected) and is not
accessible by search

Highly recommended
Discover Hidden Services and access to Tor's onion sites
Dark Web Zettabytes engines.

onionlandsearchengine.com The dark web consist

of websites that use
the public internet,
but require specific
software for access
Dark Web and are not indexed
The Dark Web is much smaller than the Deep Web and includes websites that sell drugs, weapons by search engines to
and even services of assassins. These are hidden networks avoiding their presence on the e n s u r e a n o ny m i t y.
openSurface Web, and its URLs end with .onion. These .onion domains are not indexed by regular Here stolen data is
traded, sold and use
search engines, so you can only access Dark Web content with special software called “The
for financial, political
Onion Browser,” referred to as TOR. or personal gain.

www.datawalk.com 30/37
 Internet of Things
Search Engines for the Internet of Things

www.datawalk.com 31/37
 Highly recommended
zoomeye.org

Highly recommended
Cyberspace Search Engine recording
information of devices, websites, thingful.net
services and components.

A Search Engine for the Internet of

Things. Find & use open IoT data from
around the world.

IoT

Highly recommended
shodan.io Highly recommended

iotscanner.bullguard.com
Shodan is a search engine of devices
connected to the Internet (i.e., IoT). Find
video cameras, buildings and other Check if your internet-connected
devices connected to the web. devices at home are public on Shodan.
If they are, this means they are
accessible to the public, and hackers.

www.datawalk.com 32/37
 Combine your
data with OSINT
data to detect
operations and
spot targets.

www.datawalk.com 33/37
 The ultimate solution: Use DataWalk to connect OSINT with your Example scenario (Video): Cryptocurrency
internal data. analysis
DataWalk is an Enterprise-class software platform that enables you Easily check bitcoin wallets, instantly track their transactions,
and gain insights via the darknet to identify bad actors and
to easily connect your internal data with OSINT and your other beneficial owners.
external data sources to quickly provide you with a remarkably
comprehensive view of all desired data. Being able to quickly link
together all of this data in an integrated view, coupled with powerful
tools for data analysis, can enable you to dramatically accelerate
your investigations with DataWalk.

Learn more at www.datawalk.com.

Example integrations
with OSINT subscription
services available in Example scenario (Video): Human Trafficking
DataWalk.
Leveraging indicators of human trafficking from online ads
and using DataWalk risk scoring to combine indicators with
other internal and external data sources to rank the suspects.

www.datawalk.com 34/37
 OSINT Cheat-Sheet - Page 1

Mark your Mark your Mark your

Advanced Search Engines favorites Metasearch Engines favorites
Social Media Search Engines favorites

google.com/ all-io.net social-searcher.com

advanced_search
duckduckgo.com twitterfall.com
yandex.com
zapmeta.com wiki.kenburbary.com

baidu.com webcrawler.com shadowdragon.io

Advanced search engines
search.carrot2.org
bing.com Search multiple sites at hashatit.com
once alltheinternet.com
socialbearing.com
ahmia.fi searchall.net
twitonomy.com
qwant.com infospace.com Mine and analyze social
media data, particularly followerwonk.com
myallsearch.com
Twitter
Mark your instya.com tweetbeaver.com
Internet of Things favorites
etools.ch omnisci.com
zoomeye.org
unionmetrics.com

Find video cameras, Mark your

shodan.io Deep Reconnaissance favorites hashtagify.me
buildings and other
devices connected to the spoonbill.io
web. thingful.net Gather intelligence on IP spiderfoot.net
addresses, domain names,
feed.exileed.com
e-mail addresses, names
iotscanner.bullguard.
com and more. spiderfoot.net/hx/
talkwalker.com

www.datawalk.com 35/37
36/20
 OSINT Cheat-Sheet - Page 2

Search Engines Of Domains, Emails And IP Mark your

Other OSINT Tools Mark your Mark your
Registry favorites favorites The Deep Web And The Dark Web favorites

hexillion.com mxtoolbox.com hiddenwikitor.com

Search for the hidden

services on the Tor
whois.net namechk.com ahmia.fi
network and for the deep
web links.
website.informer.com archive.org onionlandsearchengin
e.com

Looking for information on Find earlier versions of

particular domains or IP myip.ms websites, find similar similarsites.com
addresses? Take websites, and more. Mark your
advantage of search Web Scrapers favorites
engines for exploration and viewdns.info debouncer.com
investigation.
data-miner.io
If you want to download
builtwith.com viewdns.info/reverseip
large amounts of data from
particular web sites for octoparse.com
further analysis, here are
urlscan.io namecheckup.com
some tools that can help.
webscraper.io

censys.io hunch.ly

osintcombine.com/
analyzeid.com Mark your
google- analytics-id- FTP Search favorites
explorer

securitytrails.com osintcombine.com/
tiktok- quick-search globalfilesearch.com
Find a file among millions
of files located on FTP
servers.
spyonweb.com searchftps.net

www.datawalk.com virustotal.com 36/37

37/20
 OSINT Cheat-Sheet - Page 3

Search Engines Mark your Mark your Maps Mark your

Of People And Business Email Addresses favorites Cryptocurrency analysis and lookup favorites favorites

blockchain.com/ search.earthdata.nas
pipl.com a.gov/search
explorer

moneroblocks.info Type in an address or earthexplorer.usgs.g

spokeo.com ov
Search for wallets, place name, enter the
address and transactions coordinates or click the
in the cryptocurrency xmrhunter.com map to define your serach worldview.earthdata.
Looking for information on webmii.com nasa.gov
world. area.
particular people, email
addresses or telephone walletexplorer.com
numbers? Want to check skymem.info dgi.inpe.br/CDSR/
an email address of a
person of interest on the bitcoinwhoswho.com
digitalglobe.com/
basis of a domain? These connect.clearbit.com ecosystem/open-
search engines will help data
you identify hidden
content. hunter.io

voilanorbert.com

www.datawalk.com 37/37
 DataWalk is a next-generation analytical platform which
enables you to easily combine data from many different
sources (including OSINT) for instant link analysis, geospatial
analysis, visual querying, and collaborative investigations.

DataWalk would like to thank OSINT expert Jakub Karmowski

for his insights and his assistance with the content of this
guide.

For more information, visit

datawalk.com, or contact us at
[email protected].

The content provided here has been created on a best-effort basis, but DataWalk cannot guarantee the accuracy, completeness, reliability, usability or timeliness of this material. Please professionally check, or to have professionally checked, the suitability of
all content for its intended use. The contents and works in this document compiled by DataWalk are subject to copyright law. Copying, processing, distribution and any kind of use outside the limits of copyright law require the written consent of DataWalk. In
case the content is not created by DataWalk, the copyrights of third parties are being observed. In particular content of third parties are marked as such. However, if a user becomes aware of a copyright infringement, DataWalk asks for notification. Upon
www.datawalk.com notification of such violations, DataWalk will remove the content immediately.