WHOIS the Master

of 28/28
WHOIS the master an introduction to Sho'Nuff jason ross
  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of WHOIS the Master


WHOIS the masteran introduction to Sho'Nuffjason ross

about mejob: break stuff for the intrepidus group

play: with malware

poorly manage defcon group 585

refuse to use caps in slide decks (acronyms excluded)


agenda2^32 addresses ought to be enough for anybody

alphabet soup, iron fists, and ipv6

whois: awesomely full of crap

shonuff the whois master

Because IP is 32 bit, there are about 4.2 billion potential IP addresses (2^32 = 4,294,967,296). Early thought was that we'd never run out.Now there are claims that we are running out (and there have been since late 1999 / early 2000)

The address space is managed by a conglomeration of organizations, each with fun acronyms for names.Information about who the space has been assigned to is viewed as propietary, and confidential, information despite the fact that it is publicly available in the form of WHOIS.

WHOIS is useful, but has some shortcomings. Most specifically, it's difficult to query based on descriptive text, or obtain a list of "all networks that are associated with entity X".

So I wrote a tool to allow those things.3

a (very) brief history of 'the internet'lots of separate networks hooked up, some confusion ensued

InterNIC stepped out, ICANN stepped in

ICANN manages global addressing under contract to US Dept. of Commerce as IANA

(not for) profit!

The history of "the internet" as we know it today is a very convoluted (and in many cases disputed) tale.

Accordingly, i'm ignoring it entirely, and simply summarizing it as "lots of private networks came together using a suite of protocols tested by the US DOD research arm known as ARPA. Things got crazy for a bit, but everyone saw it was useful, so eventually a non profit organization was created to help things out and manage the address and name space of the network going forward."

That organization is ICANN (Internet Corporation for Assigned Names and Numbers), and they oversee the global address and namespace by managing a subsidiary organization called the Internet Assigned Numbers Authority (IANA) under contract to the US Dept. of Commerce.

In addition to global IP space assignment, IANA controls the Autonomous System (AS) Number assignments, and documents IP port number assignments made by the IETF.


ipv4 network allocationlarge blocks of addresses are allocated to global geographic regions

large blocks may be allocated to national geographic regions

blocks are divided up and allocated to local ISPs

individual addresses or small blocks are assigned to ISP customers

In general, IANA doesn't provide IP addresses to the public. It provides large blocks of space to various NICs (Network Information Centers) so that they can then be parceled out and given to ISPs in smaller chunks in a hierarchical manner.

The typical process is:* IANA assigns large blocks of addresses to Regional Internet Registries (RIR)* The RIR assigns blocks of addresses from their IANA assigned pool to National Internet Registries (NIR) or Local Internet Registries (LIR). * The RIR then assigns addresses to specific ISPs, which are then further divided into small pools for specific end user needs.5

early allocation methodsthere's so much space!

large chunks of network space allocated to single organizations

justification requirements fairly lax

zomg! this thing works!demand increased

address assignments got smaller

requirements to prove need of requested space got tighter

As the use of the internet increased, address assignments got more specific to address concerns about scarcity. Additionally, requirements for obtaining blocks of network space from RIRs grew more stringent.

The current ARIN guidelines for IP address allocation for private organizations (end-userseg. not an ISP) is:

minimum allocation: /20 (4,096 addresses)A 25% immediate utilization rate, andA 50% utilization rate within one year.



what's a RIR?Regional Internet Registry

in charge of large geographic regionsAfriNIC : AfricaAPNIC : Asia / PacificARIN : North AmericaLACNIC : Latin America & some CaribbeanRIPE NCC : Europe, Middle East, Central Asia

A RIR assigns network space to LIRs or NIRs, not to end users directly.8

what's a NIR?National Internet Registry

in charge of small geographic regions

act as an agent of the RIR

not commonly used, but there's a few

NIRs operate as an agent to the RIR which oversees them. they allocate the network space assigned to the the RIR and assign it to LIRs within their geographic region of responsibility. the NIR is not directly assigned network addresses as a RIR is, it simply manages the assignment of addresses for a specific area.9

what's a LIR?Local Internet Registry

usually an ISP

LIRs are assigned small blocks of network space, which they further divide as needed to meet the operational needs of their end users.10

why the push for ipv6?ipv4 was not designed for security

"available address space is running low"

securitymany con talks and whitepapers by folks lots smarter that i have already covered this

so i won't

scarcitythere have been comments and discussion around the fact that IPv4 space is 'running out' for years.

IEEE-USA published a report on this in 8/1999

IEEE paper on this subject can be found at: http://www.ieeeusa.org/volunteers/committees/ccp/documents/IPv6FinalwhitepaperFinalAugust2009.pdf13

the sky is falling! (aka: how low can you go?)

image taken from arstechnica: http://is.gd/dCnMM

Image taken from an article on ars technica about the scarcity of ipv4 space.14

if ipv4 is running out, where did it go?nobody that knows is telling ('freely')

nobody else knows

leading to much debate

ARIN does provide a bulk database download option. "to support the work of bona fide academic researchers, and to operators and researchers who are using the data to provide a clear benefit to the broader networking community. ARIN does not provide bulk copies of Whois data to operators who wish to incorporate this data into products, services, or internal systems with no clear benefit to the broader community."

To qualify: Step 1: Complete Request FormStep 2: Review Process"Upon receipt of your signed AUP, ARIN staff will begin a thorough review of your request by contacting you via e-mail using the e-mail address specified in the signature section of the Bulk Whois Request Form. ARIN will ask you a set of questions aimed at understanding why you require access to a bulk copy of the data."

More information on that can be found at https://www.arin.net/resources/request/bulkwhois.html


how to find outask IANA!

when that fails, ask the RIRs

then ask the LIRs

IANA maintains a list of top level block assignments at http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.xml

This file is useful for gaining a very general overview of where IP space has been assigned.

Most of the entries on the list are RIRs however, making it useless for telling where the address space is actually being used.

So, to get more meaningful information, there's a need to go down at least one level, and ask the RIRs16

overview of whois tools*nix: whois

web: http://lmgtfy.com/?q=web+whois


On most unix based systems, there's a utility called whois which can be used to perform queries of domain names and/or IP addresses.

In addition, robtex.com maintains a web based interface to whois, and a lot of other useful networking tools. 17

what's missing?no standardized outputcan't perform true wildcard querieswhois -h whois.arin.net " o . bank*"query options vary by RIRinformation is not centralized chasing referrals sucks

The whois syntax above comes close to performing a wild card query, but ARIN only supports wildcard at the end of the string.18

how accurate is whois data?contact data is required by law in most countries to be legit

ARIN is working on a policy to validate WHOIS POC info


theoretical challengesmost efficient way to scan

how to handle referrals

should i throttle queries

parsing the results

Because of how network space is allocated and assigned, often the WHOIS data obtained from the RIR points to an NSP, which places a referral to the WHOIS server run by the LIR that space has been allocated to.

If you then query the server in the referral, you can obtain more specific data about the customer the NSP has assigned the particular block to

I decided to break parsing down into multiple stages:obtain all top level WHOIS informationparse for referrals, then process them in a second stagecontinue until all referrals have been processed20

shonuff the WHOIS master!started as PHP/MySQL

then i got mocked (gently)

so i ported it to JSP/Postgresto prove it can always get worse

is now written in ruby!

whats new?better integration with shodan

privacy policy

more query types supported

linking results to shodanshodan has an API!

so i just make calls to it for youmany thanks to achillean, for letting this work!


interesting reportsorganizational breakdownwho has the most allocationswho has the most network space

geographic breakdownwhat countries have ip spacewhich countries have the most space


future plansadd in WHOIS contact datamalware IP to WHOIS correlationallows easy tie-back of malicious content to "real world" network & hosting businessesintegrate DNS records for netblocksMaltego transform?Tie-in for Fierce?Metasploit fun?

where is it?http://whoisthemaster.org

the end

@[email protected]