WHOIS the Master
Embed Size (px)
Transcript of WHOIS the Master
WHOIS the masteran introduction to Sho'Nuffjason ross
about mejob: break stuff for the intrepidus group
play: with malware
poorly manage defcon group 585
refuse to use caps in slide decks (acronyms excluded)
agenda2^32 addresses ought to be enough for anybody
alphabet soup, iron fists, and ipv6
whois: awesomely full of crap
shonuff the whois master
Because IP is 32 bit, there are about 4.2 billion potential IP addresses (2^32 = 4,294,967,296). Early thought was that we'd never run out.Now there are claims that we are running out (and there have been since late 1999 / early 2000)
The address space is managed by a conglomeration of organizations, each with fun acronyms for names.Information about who the space has been assigned to is viewed as propietary, and confidential, information despite the fact that it is publicly available in the form of WHOIS.
WHOIS is useful, but has some shortcomings. Most specifically, it's difficult to query based on descriptive text, or obtain a list of "all networks that are associated with entity X".
So I wrote a tool to allow those things.3
a (very) brief history of 'the internet'lots of separate networks hooked up, some confusion ensued
InterNIC stepped out, ICANN stepped in
ICANN manages global addressing under contract to US Dept. of Commerce as IANA
(not for) profit!
The history of "the internet" as we know it today is a very convoluted (and in many cases disputed) tale.
Accordingly, i'm ignoring it entirely, and simply summarizing it as "lots of private networks came together using a suite of protocols tested by the US DOD research arm known as ARPA. Things got crazy for a bit, but everyone saw it was useful, so eventually a non profit organization was created to help things out and manage the address and name space of the network going forward."
That organization is ICANN (Internet Corporation for Assigned Names and Numbers), and they oversee the global address and namespace by managing a subsidiary organization called the Internet Assigned Numbers Authority (IANA) under contract to the US Dept. of Commerce.
In addition to global IP space assignment, IANA controls the Autonomous System (AS) Number assignments, and documents IP port number assignments made by the IETF.
ipv4 network allocationlarge blocks of addresses are allocated to global geographic regions
large blocks may be allocated to national geographic regions
blocks are divided up and allocated to local ISPs
individual addresses or small blocks are assigned to ISP customers
In general, IANA doesn't provide IP addresses to the public. It provides large blocks of space to various NICs (Network Information Centers) so that they can then be parceled out and given to ISPs in smaller chunks in a hierarchical manner.
The typical process is:* IANA assigns large blocks of addresses to Regional Internet Registries (RIR)* The RIR assigns blocks of addresses from their IANA assigned pool to National Internet Registries (NIR) or Local Internet Registries (LIR). * The RIR then assigns addresses to specific ISPs, which are then further divided into small pools for specific end user needs.5
early allocation methodsthere's so much space!
large chunks of network space allocated to single organizations
justification requirements fairly lax
zomg! this thing works!demand increased
address assignments got smaller
requirements to prove need of requested space got tighter
As the use of the internet increased, address assignments got more specific to address concerns about scarcity. Additionally, requirements for obtaining blocks of network space from RIRs grew more stringent.
The current ARIN guidelines for IP address allocation for private organizations (end-userseg. not an ISP) is:
minimum allocation: /20 (4,096 addresses)A 25% immediate utilization rate, andA 50% utilization rate within one year.
what's a RIR?Regional Internet Registry
in charge of large geographic regionsAfriNIC : AfricaAPNIC : Asia / PacificARIN : North AmericaLACNIC : Latin America & some CaribbeanRIPE NCC : Europe, Middle East, Central Asia
A RIR assigns network space to LIRs or NIRs, not to end users directly.8
what's a NIR?National Internet Registry
in charge of small geographic regions
act as an agent of the RIR
not commonly used, but there's a few
NIRs operate as an agent to the RIR which oversees them. they allocate the network space assigned to the the RIR and assign it to LIRs within their geographic region of responsibility. the NIR is not directly assigned network addresses as a RIR is, it simply manages the assignment of addresses for a specific area.9
what's a LIR?Local Internet Registry
usually an ISP
LIRs are assigned small blocks of network space, which they further divide as needed to meet the operational needs of their end users.10
why the push for ipv6?ipv4 was not designed for security
"available address space is running low"
securitymany con talks and whitepapers by folks lots smarter that i have already covered this
so i won't
scarcitythere have been comments and discussion around the fact that IPv4 space is 'running out' for years.
IEEE-USA published a report on this in 8/1999
IEEE paper on this subject can be found at: http://www.ieeeusa.org/volunteers/committees/ccp/documents/IPv6FinalwhitepaperFinalAugust2009.pdf13
the sky is falling! (aka: how low can you go?)
image taken from arstechnica: http://is.gd/dCnMM
Image taken from an article on ars technica about the scarcity of ipv4 space.14
if ipv4 is running out, where did it go?nobody that knows is telling ('freely')
nobody else knows
leading to much debate
ARIN does provide a bulk database download option. "to support the work of bona fide academic researchers, and to operators and researchers who are using the data to provide a clear benefit to the broader networking community. ARIN does not provide bulk copies of Whois data to operators who wish to incorporate this data into products, services, or internal systems with no clear benefit to the broader community."
To qualify: Step 1: Complete Request FormStep 2: Review Process"Upon receipt of your signed AUP, ARIN staff will begin a thorough review of your request by contacting you via e-mail using the e-mail address specified in the signature section of the Bulk Whois Request Form. ARIN will ask you a set of questions aimed at understanding why you require access to a bulk copy of the data."
More information on that can be found at https://www.arin.net/resources/request/bulkwhois.html
how to find outask IANA!
when that fails, ask the RIRs
then ask the LIRs
IANA maintains a list of top level block assignments at http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.xml
This file is useful for gaining a very general overview of where IP space has been assigned.
Most of the entries on the list are RIRs however, making it useless for telling where the address space is actually being used.
So, to get more meaningful information, there's a need to go down at least one level, and ask the RIRs16
overview of whois tools*nix: whois
On most unix based systems, there's a utility called whois which can be used to perform queries of domain names and/or IP addresses.
In addition, robtex.com maintains a web based interface to whois, and a lot of other useful networking tools. 17
what's missing?no standardized outputcan't perform true wildcard querieswhois -h whois.arin.net " o . bank*"query options vary by RIRinformation is not centralized chasing referrals sucks
The whois syntax above comes close to performing a wild card query, but ARIN only supports wildcard at the end of the string.18
how accurate is whois data?contact data is required by law in most countries to be legit
ARIN is working on a policy to validate WHOIS POC info
theoretical challengesmost efficient way to scan
how to handle referrals
should i throttle queries
parsing the results
Because of how network space is allocated and assigned, often the WHOIS data obtained from the RIR points to an NSP, which places a referral to the WHOIS server run by the LIR that space has been allocated to.
If you then query the server in the referral, you can obtain more specific data about the customer the NSP has assigned the particular block to
I decided to break parsing down into multiple stages:obtain all top level WHOIS informationparse for referrals, then process them in a second stagecontinue until all referrals have been processed20
shonuff the WHOIS master!started as PHP/MySQL
then i got mocked (gently)
so i ported it to JSP/Postgresto prove it can always get worse
is now written in ruby!
whats new?better integration with shodan
more query types supported
linking results to shodanshodan has an API!
so i just make calls to it for youmany thanks to achillean, for letting this work!
interesting reportsorganizational breakdownwho has the most allocationswho has the most network space
geographic breakdownwhat countries have ip spacewhich countries have the most space
future plansadd in WHOIS contact datamalware IP to WHOIS correlationallows easy tie-back of malicious content to "real world" network & hosting businessesintegrate DNS records for netblocksMaltego transform?Tie-in for Fierce?Metasploit fun?
where is it?http://whoisthemaster.org