OSINT Basics for Attack and

Defense

By Andrew McNicol

&

Matt Foreman

Matt Foreman@s7foreman

• Security Consultant

• I have some certifications, they are made of

letters

• I do Penetration Testing, Security

Assessments, and sometimes what I call

research….

Andrew McNicol

• Security consultant

• Part-time beard developer

try:

I enjoy writing error-free Python with Google and

stackoverflow

except:pass

• I do both offensive and defensive stuff

We didn’t do it

• We are not lawyers or giving you legal advice

• We are not giving you permission or

authorizing you in any way to do anything

ever

• In fact don’t do anything ever

What is OSINT?

• OSINT has been formally defined this way…

Open-source intelligence (OSINT) is intelligence

collected from publicly available sources. In the

intelligence community (IC), the term "open" refers

to overt, publicly available sources (as opposed to

covert or clandestine sources); it is not related to

open-source software or public intelligence.

• Also check out the PTES , tons of great info

http://www.pentest-standard.org

This talk

• OSINT has been discussed from a high level to

very deep dives in past talks by others

• This talk might cover some offensive methods

of OSINT you might have seen before, but we

also want to cover some defensive uses

levering the same/similar OSINT tools that we

see mentioned less often

Shodan

• Allows users to search for publicly connected

internet devices that have been seen by Shodan• Routers

• Servers

• Firewalls and other Security Devices

• SCADA or other Control Systems… �

– This data can be searched for by IP/CIDR combo

– Open ports seen by Shodan

– Hostname, OS, Geo-Location, etc…

– Server Response

Shodan for Attackers • So it’s fairly easy to see how this can be useful to attackers.

• This simple query will show everything seen by Shodan in the

US (MERICA!) with TCP 445 open to the internet…

Shodan for Attackers

• Hopefully this an uncommon thing you would

see on engagements but you get the idea

• Without sending a packet to the end

customer/target we can identify some of their

external infrastructure and at one point what

was there

Shodan for Defenders

• Understanding what information is available in Shodan can help defenders too

• Shodan can be leveraged to fingerprint C2 servers

� Attackers sometimes make mistakes in server responses

� These unique strings could help enumerate additional C2 servers

• Can be leveraged to see server responses without actually making a request

Shodan for Defenders

• Example of searching for “Apach” and “202”:

Maltego by Paterva

• Commercially licensed

• Runs on multiple different OS

• Can integrate API’s from many different Sources

• Great for stalking people! <note> remove this its creepy </note>

• Uses various “transforms” to gather and hopefully correlate

data between various sources

Maltego for Attackers

• Here is a simple graph output of a Maltego search

Maltego for Attackers• From this point we can start mapping out infrastructure,

people, known aliases, social media, etc..

• All can be valuable information for attackers depending on the

goal…..and the scope

Maltego for Attackers• This doesn’t come with out false positives, but after enough

digging you could end out with a map like this….

Maltego for Attackers• There are many add-ons to Maltego, including one for Shodan

Maltego for Defenders

• Maltego can be a great way to perform link

analysis with indicators of compromise

• Malformity adds a lot of malware functionality:

Maltego for Defenders

• Example of running various transforms and

enumerating more information from the hash

value (mutex, C2, other samples, etc.):

Have you seen this thing, Google?• So we have all seen Google hacking before and probably the

most notable example is the Google Hacking Database or

GHDB – Originally created by Johnny Long

• And attackers obviously still use these methods today

• Here is a very simple Google search for Juniper’s SSL VPN

login page…I'm sure this was searched during the Heartbleed

craziness #heartbleedcyberAPT

Google for Attackers• This search looks for a WordPress plugin that is vulnerable to an open

redirect. About 235 results came back with modifying the query much

• exploit-db/exploits/18350/

Google for Attackers• This search looks for a search looks for open Cisco Routers, finding over 15

million results

• And here we see one of the results has an open command window

running with level 15 privileges

Google for Attackers

• People tend to reuse usernames, handles, etc...

• So if we can find some target IT personnel on a resource like

Linkedin, Facebook, or Twitter and do some searching for

common handles they like to use, sometimes you end up with

system administrators posting complete firewall

configurations onto public websites….

Google for Attackers• A little more digging on the person who shall not be named

showed that his/her username was reused on multiple sites

and one tech-help forum, which had public profiles

• This included corporate email used to register, full name, and

location

• Some users of these forums include their corporate email

signature and tagline (giving us more terms to include in

targeted searches) “We are the leader in

CyberDongleWidgets, and we know it”

• Try a Google search for some of the popular tech forums…

site:http://www.tek-tips.com/viewthread.cfm? /etc/shadow

Google for Defenders

• Knowing your organizations exposure online can help you

defend

• Google searching indicators from malware can save you time:

• Hashes, Strings, Domains/IPs, persistence mechanisms, mutexes, etc.

Google for Defenders

• Humans lie, and humans are creatures of habit:

• Fake Domain Registration Information (Emails, Phone numbers,

Addresses, etc.)

Online Data Dumps• Monitoring data dumps from the target or 2rd parties can be provide a

treasure trove of information for the attacker (Usernames, passwords,

etc.)

• From a defensive standpoint, monitoring these data dumps for your

organization can allow you to take appropriate action

Linkedin• If Social Engineering or Phishing is in scope you can

use this data to find targets

• Existing personnel to enumerate technologies and

partner relationships or company updates listing new

projects or acquisitions

• New employees are often good targets

– Minimal Training

– Don’t know IT staff on a first name basis

– Sometimes have default AD credentials (changem3)

Additional Search Resources

• Don’t put all your operators in one basket try

multiple resources

• Yandex (Russian search engine, many

operators to filter out data)

• Bing (similar to google operators but has “ip:”

option)

• Nerdydata (Indexes Code snippets, meta tags,

HTML, and JavaScript)

• Searchdiggity & FOCA (Can use API’s)

Additional Search Items

• More things to search for…

o Business Partners

o Vendor Relationships

o Are certain functions outsourced? Like HR, the

helpdesk, etc…

Wireless Communications

• Openbmap.org

• wigle.net

Find previously discovered wireless in the area of your target

Researching IPs and Domains

• Link analysis between IPs, Domains, and Name Servers can

help map out additional hostile infrastructure:

• Robtex, iplist.net, nslist.net, pop.dnstree.com, webboar.com,

centralops.net, etc.

Researching IPs and Domains

• Given a hostile Domain/IP ask yourself:

• Any fake registration information?

• What other domains point to IP?

• What other domains leverage that name server?

• What domains point to IPs around the hostile?

• Additional subdomains (skills.cnndaily.com, jobs.cnndaily.com)

• Resolve back to non-routable IP space (Loopback, bogon)

• Domains that look right, but are slightly off:

• update.macfee.com

• mirosoft.supportca.com

Researching IPs/Domains• Passive DNS can allow you to track changes to domains overtime:

•Virustotal, DNSDB, Edv-consulting

• Hostile infrastructure gets reused:

– Can help enumerate additional infrastructure

– Can assist with attribution

Automation

• Automating tasks is key – especially since you may have to do

something thousands of times

• Use Case: Whois automation with Team Cymru's Python whois

module – 1000s of lookups within seconds:

Automation

• Creating and parsing web requests via a scripting

language can save a lot of time

• Use Case: Looking up IPs via iplist.net with Python

OPSEC and OSINT• As you start digging on the line be aware of the information you

are exposing about yourself or your organization

• Many ways to control what information you give to the Internet:

• Google Cache

• Firefox Plugins:

• Foxyproxy + ssh tunneling

• User Agent Switcher

• NoScript

• Refcontrol

• Tamperdata

• Tor, VPNs, Proxy services etc.

• Separate non-attrib ISP link

Recon-ng for Attackers• Started by Tim Tomes (@LaNMaSteR53)

• Many contributors

• Menu feels similar to msfconsole

• Way too many great features to list today

• Can be a one-stop-shop to gather a ton of data

recon/hosts/gather/http/web/bing_domain

Recon-ng for Attackers

• This above example is querying searchdns.netcraft.com for additional

hosts.

• Also its worth looking at these for DNS info as well. These are querying an

DNS server of your choice instead of searching

recon/hosts/gather/dns/reverse_resolve

recon/hosts/gather/dns/brute_hosts

Recon-ng for Attackers• Search xssed.com for past entries. Can be useful for the later phases of

attack. Keep in mind the dates on some of the entries

Recon-ng for Defense

• Malwaredomainlist.com Module:

Recon-ng for Defense

• Hostname Resolver Module:

Malware Sandboxes

• Many Internet resources exist to analyze malicious samples:

� Virustotal

� Malwr.com

� ThreatExpert.com

� CWSandbox

• These are very useful, but keep in mind that they often make some of the data public

• Adversaries can monitor these online resources just like defenders

• Uploading a sample could let the adversary know you found their malware

• Cuckoo sandbox can be a free solution

Malware Sandboxes

• Cuckoo Sandbox is a free alternative to standup a

local malware sandbox:

Public doc’s and metadata

• Strings, Exiftool, etc..

• Pull down public documents (pdf, doc, ppt)

• The content itself could be as useful as

metadata

• Sometimes IT creates “how-to” guides

disclosing technology and settings used

• Metadata (What version of Office, Adobe,

etc…) When was it created and so on.

Metadata Defenders

• Can be used to extract useful strings for further research (C2,

language settings, timestamps, etc.):

– Strings, pescanner.py, Exiftool, CFF Explorer etc.

• Metadata can be used to link attacks together, and is

commonly used to name malware

• Pescanner.py:

In Summary

• OSINT is important and still gets overlooked by

attackers and defenders

• We hope that you found this talk useful

• This talk and the Python tools mentioned will

be available here shortly after the conference:

– www.primalsecurity.net