GPS-Based Geolocation of Consumer IP AddressesJames Saxon, Nick Feamster

Department of Computer Science and Center for Data and Computing,University of Chicago5730 South Ellis AveChicago, IL 60637

{jsaxon,feamster}@uchicago.edu

ABSTRACTis paper uses two commercial datasets of IP addressesfrom smartphones, geolocated through the Global Position-ing System (GPS), to characterize the geography of IP addresssubnets from mobile and broadband ISPs. Datasets that ge-olocate IP addresses based on GPS oer superlative accuracyand precision for IP geolocation and thus provide an un-precedented opportunity to understand both the accuracyof existing geolocation databases as well as other propertiesof IP addresses, such as mobility and churn. We focus ouranalysis on large cities in the United States.Aer evaluating the accuracy of existing geolocation

databases, we analyze the circumstances under which IPgeolocation databases may be more or less accurate. We ndthat geolocation databases are more accurate on xed-linethan mobile networks, that IP addresses on university net-works can be more accurately located than those from con-sumer or business networks, and that oen the paid versionsof these databases are not signicantly more accurate thanthe free versions. We then characterize how quickly subnetsassociated with xed-line networks change geographic loca-tions, and how long residential broadband ISP subscribersretain individual IP addresses. We nd, generally, that mostIP address assignments are stable over two months, althoughstability does vary across ISPs. Finally, we evaluate the suit-ability of existing IP geolocation databases for understand-ing Internet access and performance in human populationswithin specic geographies and demographics. Althoughthe median accuracy of IP geolocation is beer than 3 km insome contexts, we conclude that relying on IP geolocationdatabases to understand Internet access in densely populatedregions such as cities is premature.

1 INTRODUCTIONIP geolocation is a longstanding problem in computer net-working, with both an active academic research and a widearray of commercial solutions and applications. IP geolo-cation is used for a variety of purposes, including mappingclients to nearby content delivery network (CDN) replicas,personalization of search results and advertising, and cus-tomization of content (e.g., weather, language localization).In a legal context, IP geolocation is used for digital rights

management (e.g, geographic licensing restrictions), compli-ance with the laws and regulations of a region or country(e.g., gambling, sales taxes, privacy regulations), and to as-sist with law enforcement (e.g., determining jurisdictionsor collecting evidence). In security contexts, governmentand commercial entities use it for for counter-terrorism, at-tack aribution, monitoring access to private networks, anddetecting potential fraud. It facilitates operations and sitereliability (e.g., monitoring packet loss from a location), andinforms infrastructure investments by both industry andpolicymakers [2, 10, 12, 19, 23, 26]. Computer science re-searchers also use IP geolocation to study the properties andevolution of the network itself, such as the structure andgraph parameters of networks [6, 25].Increasingly, IP geolocation is being used to address var-

ious problems in policy and social science that entail draw-ing inferences about various demographics and geographiesbased on inferred locations of IP addresses. Social scientistshave noted the potential to use “big data” as a lens on humanbehaviors and interactions [18], and as modern society isincreasingly mediated through the Internet, many of ourinteractions are associated with IP addresses. Server logsand speed measurements, for instance, show who accessesresources and the quality of their connections. is allowsaggregate statistics or time trends. But associating thesebehaviors and network conditions with human populationsultimately requires a way to map IP addresses to physicallocations. A natural approach would be to use IP geoloca-tion with census tract-scale precision to link IP addresses tophysical locations. In this paper, we evaluate whether freeand paid IP geolocation databases can achieve this level ofaccuracy, and we explore the determinants of IP geolocationaccuracy.

e accuracy of IP geolocation databases has practical im-plications for the answers to a wide range of social and publicpolicy questions. One area of particular timeliness is thatof the so-called “digital divide.” Calls for digital equity andinclusion, already urgent, have reached a fever pitch duringthe COVID-19 pandemic. Prominent studies of broadbandperformance from Microso and M-Lab rely on IP geoloca-tion to associate Internet throughput and latencies with zipcodes [17, 22]. Ganelin and Chuang. studied whether or not

arX

iv:2

105.

1338

9v1

[cs

.NI]

27

May

202

1

§ Main Findings4.1 GPS reports are a credible groundtruth of IP locations.4.2 MaxMind’s GeoIP2 service provides the lowest median error

among tested services: 2.62 km on xed-line addresses inNYC.

5.1 IP geolocation performs beer on xed-line consumer net-works and universities, and worse on mobile broadband andbusinesses.

5.2 e physical size of subnets is correlated with the accuracywith which they are IP geolocated.

5.3 On the two-month time-scale, the median xed-line /24 IPv4subnet in US cities moves less than 1 km.

5.4 Churn of individual IP addresses on xed-line networks inmajor US cities takes months.

6 Access modalities – mobiled vs xed – dier between demo-graphic groups. Even on xed-line networks, relying on IPgeolocation to identify neighborhoods would lead to biasedresults.Table 1: Main ndings, with pointers to sections.

geolocation databases could reliably indicate socioeconomicstatus of MOOC registrants with known-physical addresses.at study ultimately concluded, as will we, that answeringsuch questions based on existing IP geolocation databases ispremature [7].

We revisit this problem now, due both to its practical impli-cations, and thanks to the availability of two highly-accurateand large-scale groundtruth datasets of GPS-located IP ad-dresses. ese datasets, from Unacast and Ookla® SpeedtestIntelligence®, aord us a view of consumer behaviors on bothxed-line and mobile networks, that is markedly dierentfrom the geolocation targets used in past work.

Table 1 presents our main ndings. e rest of this paperis organized as follows. Section 2 discusses related work inIP geolocation, both in research and in commercial productoerings. Section 3 describes the datasets that we use forthe analysis in this paper. Section 4 evaluates the quality ofthe datasets that we are using, in particular exploring thesuitability of using GPS data as a “ground truth” for evaluat-ing IP geolocation databases. Section 5 presents the resultof our study, including ndings about the circumstancesunder which IP geolocation is more or less accurate. In Sec-tion 6, we interpret and extend our results in the context ofresearch on human populations and privacy. We concludein Section 7.

2 RELATEDWORKPast work on IP geolocation generally takes three approaches,as outlined by Padmanabhan and Subramanian [21]. eirIP Geolocation work, IP2Geo, compared the complemen-tary strengths of active latency measurements (GeoPing),active traceroutes paired with DNS hints (GeoTrack), andstatic databases of outside information (GeoCluster). Each

of these approaches has evolved. Padmanabhan and Subra-manian concluded that database-driven methods held thegreatest promise. Commercial products have accordinglybuilt databases with proprietary methods that include reg-istry information, outside data, and active methods. On theother hand, academic work has tended to focus on activeand DNS-based measurements.IP geolocation methods. Starting with DNS, Spring et al.developed techniques in their Rocketfuel project to map in-frastructure (i.e., routers) to physical locations. A signicantcontribution was to optimize traceroute targets to minimizeredundancy and ensure that each path will traverse its targetISP [25], although their use of the DNS to geolocate routerswas pioneering at the time. eir subsequent approach toDNS hint identication was largely manual – “browsingthrough the list of router names” – but the resultant undnstool has proven inuential and enduring. Freedman et al.extended undns’ coverage [6]. ese projects were drivenby questions about properties of the network, specicallythe topology of large ISPs and the eciency of block assign-ments in BGP routing tables. More recently, Dan et al. [2]aempted to enumerate all possible DNS city name hintsand nalize location decisions with machine learning. LikeIP2Geo, the authors relied on a large dataset from Microsofor their ground truth, although the ground truth data wasfrom Bing instead of Hotmail.In the latency-based space, Gueye et al. [12] and Katz-

Basse et al. [15] introduced constraint-based geolocation(CBG) and topology-based geolocation (TBG). CBG is essen-tially the intersection of several latency-derived distancebuers, while TPG also localizes intermediate hosts so thattargets can be constrained by their relation to passive land-marks rather than just active probes. Subsequently, Octantincorporated both positive and negative constraints (the IPaddress is not within a certain radius) [27].In addition to this “geometric” approach are several sta-

tistical strategies. Eriksson et al. developed rst a Bayesianapproach and then a likelihood-driven choice among pos-sibilities with the CBG-derived regions [3, 4]. Other workpresents strategies using kernel density and maximum like-lihood estimation [1, 28]. It is also possible to constrain lo-cation from the covariance matrix of latency measurementswith locations.

Notable in Eriksson et al.’s Bayesian work is the insightthat outside information can help constrain or inform ge-olocation. ey used population as a measure of places’importance, as have later researchers [2]. Other forms ofinformation help as well. In trace-based work reminiscent ofTPG, Wang et al. performed extensive webscraping and anal-ysis to identify and conrm businesses with locally-hostedsites that they could “enlist” as passive landmarks. ey used

2

those landmarks to identify the locations of routers near thegeolocation target [26].Scalability has long been a limitation of active measure-

ments. Since locations are most-constrained by the closestlocations, Hu et al. developed methods to prioritize measure-ments from nearby hosts, eectively by localizing avatarsfrom subnets [13]. Alternatively, Li et al. “ip” the standardinfrastructure of active geolocation with GeoGet: the tar-gets to be localized measure the latency themselves, throughjavascript, rather than generating pings through an API [19].is reduces the number of servers and trac required, andit is also helpful since clients’ devices or networks may failto respond to pings or complete traceroutes.Evaluating commercial services. ese advancesnotwithstanding, commercial geolocation tends to be imple-mented through databases, which are inexpensive to distrib-ute and can aggregate historical observations across manysources. e leading services—MaxMind, IP2Location, Aka-mai, or NetAcuity—all use proprietary methods. A numberof papers assess the performance of these databases, com-paring with the preceding active methods [11], points-of-presence paired with routing tables from a large ISP [23],DNS lookups paired with ground truth rules from domainoperators [8], from RIPE ATLAS built-in measurements, orPlanetLab nodes, or against each other, sometimes with a ma-jority logic applied. e databases are themselves oen takenas the ground truth for latency-based measurements oenwith a sort of majority logic. Shavi and Zilberman employthat strategy in evaluating the databases themselves, but alsofocus on consistency among addresses determined to sharea point-of-presence, based on an earlier algorithm [5, 24].Similarly, Huaker et al. assess the agreement of countrydeterminations and distances from a centroid, from majorityvotes (supplemented by PlanetLab ground-truth and limitedround-trip time measurements) [14].On the whole, both the formal literature and “popular

wisdom” paint a fairly pessimistic picture of geolocationperformance. Research studies from about ten years ago as-sessed median accuracy of these services at 25 km inWesternEurope and 100 km in the United States. On the commercialside, Poese et al. quote median accuracies between tens andhundreds of kilometers for MaxMind and IP2Location [23].Other early works present distributions with ranges betweenhundreds or thousands of kilometers [24]. Gharaibeh et al.present results for routers in particular, with median ac-curacies between 10 km for NetAcuity and 1,000 km forIP2Location, on either extreme of the free and paid versionsof MaxMind. More recently, Dan et al. presented mediansbetween 10 and 30 km, depending on the sample and service.[2]ey present results in 10 km bins and do not dierentiateperformance at the very boom of the range.

Studies of how Internet infrastructure aects geoloca-tion accuracy. A persistent though somewhat more subtlecurrent of the literature has explored the physical struc-ture of the Internet and its relation to geolocation accuracy.Padmanabhan and Subramanian anticipated the interplaybetween network infrastructure and geolocation accuracyin 2001 [21]. ey noted the impact of the geographical con-centration of AOL’s login nodes on accuracy, and showedthat clusters of addresses that were physically larger wereassociated with poorer performance for the GeoCluster (data-base) method. is point was echoed in 2007 Gueye et al.[11] Similarly, Freedman et al. measured the physical scaleof autonomous systems. Later, Gharaibeh et al. probed thecommon assumption of databases that /24 subnets are co-located [11] ose papers show that systems, subnets, andIP prexes advertised by the Border Gateway Protocol (BGP)can span large physical distances. In this paper, we seek toextend this work, aiming to identify the circumstances whenthey are large or small. Huaker et al. characterized accuracyaccording to carriers’ network role; we extend that line ofexploration in this research, exploring how accuracy variesbetween commercial ISPs, large companies, and universi-ties. We categorize addresses by “Doing-Business As” namesreported in IP address registries; to our knowledge, such acharacterization is unprecedented, at least in the current erawhere mobile devices are signicantly more prevalent thanthey were a decade ago.In addition to work on IP address locations, our data also

shed light on the persistence of dynamically assigned IPaddresses, itself an active area of analysis. Recent worksusing RIPE Atlas probes [16] and browser extensions [20]have found retention times on the order of days, but weobserve a signicantly slower rate of churn than observedin related contemporaneous work, suggesting that questionsabout the persistence of IP address assignment continues todeserve aention.How this paper extends past work. Past work that eval-uates IP geolocation accuracy has tended to rely either onactive measurements of somewhat coarse precision, or ona fairly consistent set of (unrepresentative) benchmarks:specically, PlanetLab sites and university clusters. edataset we rely on for this paper of course has its ownpeculiarities—it is a non-random sample of mobile devices—but this view from the access network, including mobile de-vices, is critical and distinctive from past studies. It is a largesample, indicative of realistic consumer geolocation targetsin major cities in the United States. e Global PositioningSystem (GPS) has long served as a counterpoint to IP geolo-cation, both as a benchmark of accuracy and as an analogin multilateration. Historically, its deployment and use forInternet measurement felt impossibly far o [3, 4, 15], butthe future has now arrived.

3

Geography Date # of IPs

Unacast Clusters NYC, Chi., Phl Aug-Oct 2020 248M+ 40 km buer Apr 2021 9.5M

Speedtest Intelligence Chicago Region 2020 4M

MaxMind Free Global Aug 2020 -Global Apr 2021 -

MaxMind Paid North America Apr 2021 -

IP2Location Free Global Aug 2020 -Global Apr 2021 -

IP2Location Paid Global Apr 2021 -Table 2: Data sources: geographic and temporal coverage, anddata volumes (for GPS data only).

is paper complements and extends previous work asa result of its large sample of consumer smartphone loca-tions on diverse networks. e primary dataset was providedby Unacast; we conrm our basic ndings with a smaller,Chicago-only sample of GPS-located Speedtest® data fromOokla®. Similar datasets are readily available for commercialapplications and academic research. We exploit this sampleto understand how IP geolocation accuracy varies by geog-raphy, carrier, mode of access, and other factors. In contrastto previous work, which has tended to question the over-all reliability of geolocation even at country-level accuracy,we show that it works fairly well in predictable and well-dened contexts. Nevertheless, the imperfect accuracy andcontext-specic performance still currently constrain theapplicability of IP geolocation for studying Internet accessby human populations.

3 THE DATAis paper relies on two commercial datasets with GPS-tagged IP addresses to analyze the geography of consumerIP addresses. We also evaluate and analyze the performanceof databases for IP geolocation from two popular, commer-cial services: IP2Location and MaxMind from the same timeperiods. Table 2 outlines the datasets that we use in ouranalysis, and Figure 1 explains how these datasets are joinedand augmented in our analysis.e GPS data were delivered anonymized and remain so.

e data were collected in accordance with local laws andopt-out policies (GDPR), and analyzed with approval fromour university’s Institutional Review Board (IRB). e IRBapproved analysis of reconstructed “home locations” for ear-lier work, but emphasized the sensitivity of doing so. Forthat reason, we avoided geographic analysis of individualdevices in this project, and proxied “residence” simply asactivities recorded at night.

device_idtimecluster_typedurationgps_latgps_lonip_addrnighttractdbadb_latdb_londist_gps_db

Unacast Betweenmidnightand 6am

VincentyDistance

WhoisMaxMind/IP2Location

SpatialJoin

Figure 1: Simplied illustration of the data augmentationprocess, for Unacast data. e fundamental data consist ofdevice identiers, times, locations, and IP addresses. Clusters(see text) are also labelled by type, for instance, TRAVEL orLONG AREA DWELL. e time and duration are used to constructa ag for night-time clusters. e IP address is used with theARIN whois resource to construct Doing Business As (DBA)names, and database-dened locations are retrieved from upto four databases by MaxMind and IP2Location. Vincentydistances are calculated between database and GPS locations

3.1 Unacast GPS Smartphone Locationse primary dataset used for the analysis is from Unacast,a location intelligence rm. is dataset contains GPS loca-tions reported by mobile devices, along with timestamps andunique, anonymous identiers. Unacast aggregates multiplelocation data streams from other rms; they perform exten-sive data validation, de-duplication, and processing on thosestreams. e exact applications that generate locations arenot provided. e share of data reporting IANA reservedor private addresses low, 0.5%, and the share of addressesassociated with foreign Internet registries totals just 0.2%(mostly RIPE, breakdown shown in the Appendix). e traf-c observed in the Unacast dataset is overwhelmingly IPv4,at 99.6%.We use data contained within a 40 mile radius of three

major cities in the United States: New York, Chicago, andPhiladelphia. is large buer includes both urban and ruralpopulations. Two samples were provided: e rst is fromAugust–October 2020. A second, shorter period from April2021 is aligned with licenses for paid geolocation databasesto allow us to evaluate the accuracy of those services. As dis-cussed below, the IP address from which a physical locationis reported is recorded for about half of clusters in the 2020sample, although this falls to just 15% in the 2021 sample.Data are used only when they contain an IP address, so thefull dataset thus oers IP addresses recorded at over 248 mil-lion locations. e median reported GPS location accuracyis 17 meters on the 2020 sample, and 11 meters on the 2021sample. A small fraction of data (1.7%) are recorded with only

4

four decimal points of coordinate precision, correspondingto about 10 m; we exclude them from subsequent analyses,along with any with estimated device accuracy greater than50 m. We also exclude the small fraction of addresses fromreserved IP ranges and foreign NICs.Each line of data represents a cluster of location reports,

called bumps. Clusters are built by combining bumps froman individual device that are close in both time and space,using Unacast’s proprietary algorithm. at algorithm usesmachine learning to account for variation in physical scaleamong locations: a mall is larger than a coee shop or a home.Clusters are labelled according to their durations, which arealso reported. Locations recorded during movement arelabelled as TRAVEL. See the Appendix for a listing of clusterfrequencies. is clustering reduces the data volume by afactor of 20 while retaining most of the information. Justas important, the raw, un-clustered data oen cannot berelicensed.e clustering entails some subtlety: a single physical

location and IP address is reported per cluster, and thus thecentroid of a TRAVEL cluster may not exactly coincide withthe moment that the reported IP address was used. Indeed,the physical location of a consumer IP address is oen notxed; for instance, consumers can roam freely through theirhome while connected to their WiFi. In practice, individualIP addresses are recorded at many physical locations—andthese locations may be close or distant from each other.As a means of selecting residential IP addresses, we ag

clusters generated at night. Night-time clusters are those forwhich the period between the rst and last bumps extendsinto the hours between midnight to 6am of any day. eseclusters represent just 4.7% of clusters but 26% of bumps.Only 18% of devices have at least one night-time cluster, butthose devices generate the vast majority of the data: 80%of clusters and 88% of bumps. In short, weighted by datavolume, most devices have observations at times when theycan reasonably be assumed to be at home. For the set ofdevices with night-time clusters, the ratio of devices to thepopulation of the study region is about one device for every20 people.To investigate the determinants of geolocation accuracy,

we also identify ISPs. Each address is associated with its/24 subnet, whose organization is retrieved from the ARINwhois registry, on September 1 2020, or April 25 2021. If theprex size exceeds 24 on IPv4 or 48 on IPv6, we follow thelink to the “parent” network. is strategy is philosophicallysimilar to an ASN lookup, and diers in practice primar-ily in superior coverage of the Department of Defense NIC,wireless carriers, and foreign NICs. e ASN lookup also“fractures” organizations like small city governments or busi-nesses from their providers. We associate large and commonorganizations with standardized “Doing Business As” (DBA)

names, taking particular care to capture the major ISPs ineach market (Comcast, Charter, etc.). We separate AT&T’sand Verizon’s mobile broadband from their xed oeringsbased on the words “Mobility” or “Wireless” in the organi-zation name. is may not be a perfect division: “VerizonBusiness” and “AT&T Services” may include mobile oer-ings, but examining the ASN tables suggests this is not theirprimary use. It is worth noting that the sample is domi-nated by locations recorded while connected through mobileproviders: there are ten times as many addresses on AT&Tmobile than AT&T xed-line services, and more than vetimes as many on Verizon mobile than Verizon xed-line.However, as we will separate addresses by ISP, this samplevolume eect is largely “partitioned out.” Ultimately, eachaddress is associated with a single DBA name for analysis.

ese procedures also identify large companies and insti-tutions, in particular, universities. We ag addresses fromuniversities with at least ten thousand students, and Fortune100 companies. University clusters are “classic” targets foracademic work on geolocation, since they have have mean-ingful and well-known locations, but they are not represen-tative of the consumer space. We exclude ISPs, includingGoogle, from the Fortune 100 set. We tabulate IANA specialuse and non-ARIN addresses, as checks on the underlyingdata, but exclude these from subsequent analysis.

3.2 Geolocated Ookla Speedtest DataIn addition to the data from Unacast, we have obtainedSpeedtest data from Ookla. e data are for tests performedon smartphones, again with locations from GPS. is datasetis substantially smaller, and is limited in geographic extentto the counties surrounding Chicago. We appeal to thesedata as a cross-check of the Unacast data that, though morevoluminous, were not designed for this work.

We have received over 4 million individual Speedtest mea-surements for 2020, though only 270 thousand match theperiod of the study (August 2020). Unlike Unacast data, eachlocation comes from a single moment in time (it is not a clus-ter). On the other hand, the Speedtest data include only therst three bytes of the IP address, due to privacy restrictions.We rely on Ookla’s coding of Internet Service Providers.

3.3 Geolocation Databases and DistancesWe obtain the free versions of the MaxMind and IP2Locationdatabases, for August 1, 2020. We also acquire both thefree and paid versions of these databases, from from April26, 2021. e NetAcuity and Akamai geolocation services,which are much more expensive, are not included in thiswork. Using these databases, we geolocate IP addresses fromthe GPS sample. Per the license, this is done only for themonths of GPS data matching the databases (August 2020and April 2021).

5

We thenmeasure the Vincenty distance (on the ellipsoid ofEarth) from each IP-geolocated point to the location recordedby the GPS-enabled device. For most of what follows, wetake the centroids of the GPS clusters as the “ground truth”and call the entire distance the “accuracy” or “error.” Sincethe database providers acknowledge their limited resolutionand in certain cases quantify it accurately, this languageis perhaps unfair: it is dierent for a database to acknowl-edge a location as unknown or indeterminate (as in reserved,private addresses) than to be “wrong” about the location.Moreover, the GPS data themselves do have some limita-tions, noted below. Semantics aside, the balance of this worktabulates distances with respect to the ground truth andseeks to explain their heterogeneity.

4 EVALUATING DATA QUALITYBefore coming to questions about the properties of consumerIP addresses, we analyze the quality of our data. We rstexplore the consistency of the GPS-based location data we ob-tain from Unacast and Ookla by comparing the data againsteach other, with respect to geolocation databases.

4.1 Are GPS data a credible ground truth ofIP address locations?

e accuracy of IP geolocation is central to Unacast’s corebusiness, and the company dedicates enormous resourcesto validating and maintaining their incoming data streams.While GPS data from smartphones is generally understoodto be accurate, datasets from smartphone-based servicesdo oen incorporate additional data to assist with locatingdevices in circumstances where GPS does not work (e.g.,indoors). us, while we expect these GPS-based datasets tobe reasonably accurate in general, it behooves us to explorethe quality of these datasets before proceeding with otherquestions. Since we aim to use these datasets as “groundtruth”, this analysis may seem a bit of circular. Our strategyis to compare the consistency of IP geolocation result fordierent GPS contexts and across wholely independent GPSsamples (Unacast and Ookla). Of course, this analysis doesnot exclude the possibility of systematic errors arising inboth GPS datasets, or across all datasets, but given the lack offurther ground truths, we are le with consistency checks.Evaluating cluster types. e correspondence betweenGPS coordinates and the physical location of its IP addressmay not be perfect. For example, we expect that the cluster-ing procedures could aect the “compatibility” of the IP ad-dress and GPS location. Further, if a GPS location is recordedwhen no network is available, it may be subsequently re-ported at a dierent physical location where an IP addresscan be obtained. We would expect these eects to be mostsevere for TRAVEL clusters, as previously discussed. e ip

10 1 100 101 102 1030.0

0.2

0.4

0.6

0.8

1.0

CDF:

Max

Min

d

AT&TComcastRCNWOW

10 1 100 101 102 103

Geolocation Error [km]0.0

0.2

0.4

0.6

0.8

1.0

CDF:

IP2L

oc.

UnacastOokla

Figure 2: Geolocation error of GPS location targets in Chicago,on both Unacast and Ookla Speedtest Intelligence® data,1 usingthe free versions of the MaxMind and IP2Location databasesfor August 2020.

side of this argument is that navigation applications are morelikely to be active during TRAVEL. ese apps record locationmore frequently, which could improve accuracy.

To evaluate the eects of imperfect knowledge of locations,stemming from these eects, we contrast TRAVEL clusterswith others. We will show below that geolocation perfor-mance diers by network. Obviously, it is easier to “travel”when connected to a mobile than xed-line network. Wetherefore focus this analysis on a single, mobile network:AT&T Mobility. We do observe that accuracy is worse fortravel than non-travel data, but the dierence at the medianis only about 2.5%, for either IP2Location or MaxMind. Ascan be seen in the Appendix, the cumulative distributionfunctions for travel and non-travel clusters are fairly acrosstheir entire domain.Analysis of independent samples. To further validatethe GPS data, we contrast data from Unacast with Ookla,for xed-line broadband ISPs, in Chicago and August 2020,where both datasets are available and aligned with the freeversions of the geolocation databases. Figure 2 shows theseresults. MaxMind performs somewhat beer on Comcastaddresses from the Ookla dataset than the Unacast data, and

1Based on the authors’ analysis of Ookla® Speedtest Intelligence® data forAugust 2020 in Chicago. Ookla trademarks used under license and reprintedwith permission.

6

somewhat worse on AT&T; RCN and WOW! are very con-sistent. Discrepancies are somewhat larger on IP2Locationas is comparative performance by the two databases.

One notable feature in the 2020 Unacast dataset is a smallbut non-negligible share of the data with IP geolocation “er-ror” very close to zero. Depending on the ISP, that shareis 4-5% of the xed-line locations on MaxMind and 1-2% ofthose on on IP2Location. On close inspection, these appearto be locations reported by applications relying the IP Geolo-cation services themselves, rather than true GPS coordinates.For example, these ultra-“accurate” locations are not at resi-dences, as one might expect for xed-line ISPs, but in parks,as is MaxMind’s practice for default locations. [16, 20] eshare of “too-close” locations is smaller on the 2021 clusters;however, the IP address eld is populated for a lower shareof those data.However, the basic features of Figure 2 are consistent in

the completely separate sample from Ookla, which does notexhibit this feature.

4.2 Which database provides the lowesterror in location?

e practical question is which database to use, and howwellit should be expected to perform. is analysis, uniquely, isperformed using the April 2021 sample from Unacast, forwhich the paid geolocation databases were licensed. SinceSection 5.1 will show that geolocation on mobile broadbandis very poor, this analysis focusses on xed-line broadband.

e short answer is that MaxMind’s paid database, GeoIP2,provides the best accuracy, in terms of geolocation error onall quantiles. e traditional way of reporting this is themedian error, which is 2.62 km in New York City, 3.31 kmin Chicago, and 4.02 km in Philadelphia. Other quantilesand the other three databases are shown in Table 3. Figure 3shows the distribution of distances by city and database. Weuse “city” to refer to the city itself along with the 40-milebuer around it. Because the distance from Staten Island toNorth Philadelphia is only 46 miles, some data are includedin the curves for both New York and Philadelphia.

Although the paid databases are more accurate in each cityand at every quantile, the relative improvements in accuracyare modest. An important limitation of this particular studyis our focus on urban areas in the United States. In particular,we do not test accuracy of these databases outside of majormetro areas, and global or national performance may ofcourse be dierent. Nonetheless, it would be possible toperform the analysis we have presented in this section forother datasets, if and when they are made available.

New YorkMaxMind IP2Location

antiles Paid Free Paid Free0.10 0.73 0.78 2.96 3.040.25 1.37 1.47 6.06 6.140.50 2.62 2.82 12.04 12.140.75 4.95 5.49 30.05 30.480.90 9.66 11.02 61.84 63.01

ChicagoMaxMind IP2Location

antiles Paid Free Paid Free0.10 0.97 1.02 4.58 4.730.25 1.75 1.86 9.79 9.910.50 3.31 3.57 23.95 24.280.75 6.42 7.10 45.58 45.730.90 13.04 16.09 196.43 202.85

PhiladelphiaMaxMind IP2Location

antiles Paid Free Paid Free0.10 1.13 1.20 4.16 4.190.25 2.14 2.27 8.97 9.010.50 4.02 4.29 20.87 20.990.75 7.57 8.23 39.60 39.710.90 13.53 15.03 78.34 78.79

Table 3: antiles of accuracy in kilometers, for each databaseand city.

5 THE GEOGRAPHY OF CONSUMERSUBNETS

We now turn from an initial assessment of the dataset anddatabases, to measurements of the geography of the under-lying networks.

5.1 Under what circumstances are IPgeolocation databases accurate?

e basic results of Section 4.2 mask extreme but unsurpris-ing heterogeneity. Figure 3 already shows that geolocationperforms beer in New York than Chicago, and beer inChicago than Philadelphia. But the largest source of hetero-geneity stems from providers, which deploy dierent phys-ical infrastructures (and serve dierent cities). is entireSection relies entirely on the free databases.Fixed-line and mobile networks. Figure 4 shows accura-cies observed in New York, Chicago, and Philadelphia formajor broadband carriers in each market. In the best cases,such as either RCN or Comcast on MaxMind in Chicago, themedian error is less than 5 km. In each city/database pair, the

7

0.0

0.5

1.0CD

FNew York

MaxMind PaidMaxMindIP2Location PaidIP2Location

0.0

0.5

1.0

CDF

Chicago

10 1 100 101 102 103

Geolocation Error [km]0.0

0.5

1.0

CDF

Philadelphia

Figure 3: Cumulative distribution function by geolocationdatabase and city. Colors reference databases, and line stylesdenote paid and free versions.

10 1 101 1030.0

0.5

1.0

CDF:

Max

Min

d

New York

10 1 101 103

Chicago

AT&TCablevisionCharter

ComcastRCNSprint

T-MobileVerizonWOW

10 1 101 103

Philadelphia

10 1 101 1030.0

0.5

1.0

CDF:

IP2L

oc.

10 1 101 103

Geolocation Error [km]10 1 101 103

Figure 4: Geolocation performance by city, database provider,and ISP. Free versions of the database are used in each case.ISPs are shown by their “brand” colors, according to the whoisdatabase, which leaves the Sprint and T-Mobile networks dis-tinguishable. Fixed-line networks are denoted by solid lineswhile mobile networks shown by dashed lines.

accuracy is good for xed broadband and poor for any mo-bile broadband. In Chicago, MaxMind is more accurate withxed-line (AT&T, RCN, WOW, and Comcast) than on mobile(AT&T Mobile, T-Mobile, Sprint, Verizon Mobile) carriers.(IP2Location performs poorly with RCN.) Similarly in NewYork, Charter, Cablevision, Comcast and Verizon are beer

localized than AT&T Mobile, Sprint, T-Mobile, and VerizonMobile; and in Philadelphia, geolocation is more accurate onComcast than Verizon, T-Mobile, AT&T Mobile, or VerizonMobile.antitatively, the share of Comcast addresses in New

York that MaxMind’s free service locates within 10 km ofthe GPS location is 66%. At the other extreme, 87% of T-Mobile addresses from the New York region are assigned tojust two distinct locations representing New York itself andNewark; 98% are assigned either to those two, or to one of sixother locations in Philadelphia (3), Providence, Boston, andWashington. As a result, only 18% of devices are assignedwithin 10 km of their true location. In fairness, it must beemphasized that MaxMind does not claim to assign thesedevices within 10 km: almost all of the T-Mobile addressesassigned to the New York and Newark locations are in the200 km accuracy class.

is basic dichotomy between mobile and xed broadbandis apparent even within ISPs. AT&T oers both services inChicago, and the CDFs for its xed-line and mobile servicesare widely separated. e individual subnets with the largestgeolocation errors all belong to the AT&T Mobility organi-zation. In New York and Philadelphia, AT&T only operatesmobile networks, and this is reected in those cumulativedistributions. e observation that mobile and xed-linenetworks dier may appear obvious once stated, but it neednot have been true. Mobile carriers could have constructednetworks so that individual antennas held a xed set of IPaddresses. at does not appear to be what they did.Universities, businesses, and consumer networks. Be-fore continuing, we also contrast geolocation performanceon consumer xed-broadband, with large universities andcompanies. We include universities with at least ten thou-sand students, and Fortune 100 companies other than ISPs.Again, we note that we are implicitly studying the WiFi ac-cess points that these institutions operate andwhich their em-ployees, students, and clients connect to via mobile devices,rather than wired connections or xed infrastructures ofservers. Universities are a classic target in the academic liter-ature on geolocation, but Figure 5 shows that they are in gen-eral more-accurately geolocated than either consumer ISPs orcompanies. is is not surprising: they have large, physically-concentrated networks, with registration addresses clearlyspelled out in ARIN records. In most cases, median geoloca-tion error on MaxMind (free) is less than 2 km, though a fewinstitutes – DePaul in Chicago and the City University ofNew York – are mislocated by upwards of 10 km. Note thatthe nominal sample period is August 2020, when students –and indeed many sta and faculty – were not on campus,due to both summer vacation and the coronavirus pandemic.Figures 3-5 suggest that for a substantial share of IP ad-

dresses, IP geolocation is quite accurate. However, this does8

10 1 101 1030.00

0.25

0.50

0.75

1.00

CDF:

Max

Min

d

New York

10 1 101 103

Chicago

Fixed ISPs Universities Fortune 100

10 1 101 103

Philadelphia

10 1 101 1030.00

0.25

0.50

0.75

1.00

CDF:

IP2L

oc.

10 1 101 103

Geolocation Error [km]10 1 101 103

Figure 5: Geolocation performance on consumer ISPs, con-trasted with large universities and Fortune 100 companies.

not do us much good unless those locations can be identi-ed in advance. It is already clear that the picture is rosierwith xed broadband. ose data can be easily identied,either via a whois look-up or (in some cases) through thegeolocation databases themselves. But mobile and xed isnot the only lever. MaxMind is able to perform beer onRCN than on Comcast in Chicago, and beer on Charteror Cablevision than Comcast in New York. How are we toidentify localizable blocks of addresses?We highlight two additional methods. MaxMind’s data-

base provides an “accuracy” eld that successfully identiesthe precision of entries. Figure 6 shows the CDF for suc-cessive bins of claimed accuracy on the free database. Inthe most precise bin, accuracy of “1 km,” the median devicein Chicago is geolocated just 2.0 km from the GPS-basedlocation. e “error” with respect to the ground-truth de-grades in-line with quoted accuracy, though there is enor-mous spread in the least-precise, 500 km bin. It is thus possi-ble to identify accurately-located addresses – MaxMind doesit. But this leaves an open question: why are those addresseswell or ill-located?

at brings us to the second method. Our hypothesisis that if /24 subnets are geographically localized – small –then addresses within them are more-likely to be accuratelygeolocated. If they are large, then precise locations wouldrequire ner address-level data. e question can then bere-posed: how large are subnets, and is their size in factcorrelated with geolocation accuracy?

5.2 What is the geographic scale of /24subnets?

What are the physical and network properties of accurately-located subnets? We focus this analysis on a single, xednetwork – Comcast – and require that subnets have at least

10 1 100 101 102 103

Geolocation Error [km]0.0

0.2

0.4

0.6

0.8

1.0

Cum

ulat

ive

Dist

ribut

ion MaxMind

Accuracy1 km5 km10 km20 km50 km100 km200 km500 km

Figure 6: Cumulative distribution of geolocation accuracy onthe MaxMind database, by quoted accuracy bin.

Figure 7: Illustration of the procedure dening subnet scales,for one dispersed and one well-localized subnet in Chicago.Convex hulls wrap around 𝑓 = 0.9 of the points within thesubnet. e “scale” is the square root of this area. e linearscale on the right-hand side (67.176.158.0/24) is a factor of 8larger than on the right-hand side. Gaussian noise has beenadded to the locations for illustrative purposes only.

10 devices and 10 distinct IP addresses. ere are over twentythousand such subnets between the three cities.Constructing a physical scale. To quantify whether ornot a subnet is localized, we dene a characteristic physi-cal scale. Many subnets have some outliers, perhaps withlocations reported aer the fact. To mitigate the impact ofthese outliers, we must rst identify them. We compute themedioid of locations in the subnet, dened in this case sim-ply as the median of the 𝑥 and 𝑦 coordinates in a projected(at) geometry (EPSG 2163). We then measure individuallocations’ distances from that medioid. We select a cong-urable fraction 𝑓 of the data that is “closest” by that measure.For that subset of the data, we calculate the convex hull. If𝑓 = 1, then the convex hull covers all locations recorded onthe subnet; if 𝑓 = 1/2, it covers the half of points closest tothe medioid. Finally we take the area of the convex hull, and“convert” this area to a distance by taking its square root.at square root denes the length scale of the subnet. Fig-ure 7 illustrates this procedure for two subnets. (To preserveanonymity, random noise has been added to the individualpoints in the illustration.)Figure 8 shows this distance scale for subnets with at

least 10 devices and addresses, for several choices of 𝑓 . By9

10 1 100 101 102

Subnet Scale [km]0.0

0.2

0.4

0.6

0.8

1.0Cu

mul

ativ

e Di

strib

utio

n Fraction ofPoints in Hull

0.250.500.750.901.00

Figure 8: Cumulative density of subnets’ distance scale asderived from the convex hull of locations, as described in thetext.

construction, the scale is smaller or larger when outliers aremore or less suppressed, respectively. Seing 𝑓 = 0.5 resultsin a median subnet scale of 4.3 km, and 𝑓 = 0.9 leads to ascale of 9.9 km. However, the proportion of subnets withscales exceeding 10 km is small for any choice of 𝑓 < 0.9.e relationship of physical scale and accuracy. Armedwith this scale, we return to the earlier question: when cansubnets be accurately located? Discarding locations with ge-olocation error over 100 km, the correlation between 𝑓 = 0.75subnet scale and mean geolocation error, is 0.69 for MaxMindFree (GeoLite) but only 0.30 on IP2Location (which has worseoverall performance). We thus conrm the hypothesis thatlocalization and localizability are related, though strictlyspeaking, this analysis is not causal.

Still, this analysis has delayed but not answered the ques-tion; it suggests that geolocation fails on xed-line addresseswhen subnets are large, which raises in turn the issue of whylarge subnets exist at all. Comcast uses both large and smallsubnets. Are large ones used dierently?Taking a hint from the results on mobile broadband, we

hypothesize that the small subnets are nearly static whereaslarge ones provide a reserve of “ephemeral” addresses –perhaps for devices waiting for a static one. A client assignedto an “ephemeral” address would be unlikely to fall on thatsame address again, whereas a “sticky” address granted toa home network would be used repeatedly. e relevantvariable is thus the number of times that a single client isobserved at each IP address (weighted by visits). Figure 9conrms the hypothesis: for subnets with scale greater than20 km (𝑓 = 0.75) nearly half of visitors to an IP address visitexactly once.

5.3 How persistent are the physicallocations of /24 subnets?

Geolocation providers are quick to point out that databasesevolve continuously. Clearly, the physical infrastructure

0 10 20 30 40Number of Times Device Accesses Address

0.0

0.1

0.2

0.3

0.4

Dens

ity o

f Clu

ster

s Scale < 20 kmScale > 20 km

Figure 9: e number of times a single device visits a singleIP address on the subnet (weighted by visits). On subnets withscale greater than 20 km (𝑓 = 0.75), over half of visits device/IPpairs are unique.

10 2 10 1 100 101

Subnet Movement [km],August to October 2020

0.0

0.2

0.4

0.6

0.8

1.0

Cum

ulat

ive

Dist

ribut

ion ATT

CablevisionCharterComcastRCNWOW

Figure 10: Distance moved by the medioids of /24 subnet onxed-line networks, over a two-month period from August toOctober 2020.

of the Internet evolves over time, but how quickly do sub-nets actually move? Because mobile networks subnets arealready physically very large, and addresses on them arenot accurately located, we focus this analysis on xed-linebroadband.emovement of subnets. Figure 10 presents the physicaldistance between the medioids of individual /24 subnets, asconstructed in August and October 2020. As in Section 5.2,the medioid is the median of the 𝑥 and 𝑦 coordinates. Toenter into this gure, subnets must have at least ten uniquedevices and ten unique addresses in eachmonth. We consideronly xed-line broadband carriers, for this exercise.On each network considered, the median subnet moves

less than a kilometer; ere is some inherent variabilityin our construction of the medioid as the “location” of thesubnet in each period, and the Figure shows the dierenceof these two “noisy” measurements. We thus suspect thatthis overstates movement. In short, we conclude that on thistime scale, subnet locations are quite stableIs the sample biased? A substantial threat to this analysisis sample composition: by requiring 10 devices and 10 ad-dresses, the subnet must be observed in New York, Chicago,

10

or Philadelphia in both months, to enter the sample at all.However, it does not seem to be the case that subnets aremoving out of sample. Of the subnets satisfying the cutsin August, 92% also pass them in October (vice versa, 96%).If we raise the thresholds to enter the sample, requiring 20devices and 20 addresses, 95% of subnets passing these cutsin August also show up with at least 10 devices in October(vice versa, 98%). Raising the thresholds yet further to 50devices and 50 addresses, the persistence from August toOctober exceeds 99% (vice versa, 98%).

5.4 How long does a consumer connectionretain an IP address?

e analyses above show that IP addresses identify physicallocations at the level of 2 km, under the best circumstances.On its own, the IP address clearly does not identify individu-als.Of course, physical locations – geographic coordinates –

are not the only way in which IP addresses identify people.Linked to log-ins or other online behaviors, IP addresses canbe used to track users over time even without cookies orngerprinting (or as a component of a ngerprint). If theIP address is static for a long time, it easier to link onlinebehaviors. A critical concern is thus how long xed-line IPaddresses remain with a single household.Dening churn. We dene churn as the likelihood of adevice returning to the same IP address on an ISP, aer adelay of 𝑑 days. e denominator includes every pair ofnight-time connections by a single device to one ISP, 𝑑 daysapart. We select night-time activity, to focus on periodswhen devices can be reasonably assumed “at home.” enumerator is the number of those pairs for which the twonights’ connections are on the same IP address. Stated lessformally: if I see a device on Monday night (𝑑 = 0) and againon the same ISP Tuesday night (𝑑 = 1), what are the chancesthat it will be on the same IP address? What about nextMonday (𝑑 = 7)?

Since the sample selection is somewhat peculiar – devicesare necessarily recorded on xed-line broadband on multiplenights – one should take some care in interpreting theseresults. is consideration is particularly acute at the max-imum of the range, since there are fewer opportunities fora device to be observed 80 days apart (just 10) let alone 90(just 1). is perhaps explains the drop-o on the right-handside.Rates of change, over two months. Figure 11 shows thepersistence of IP-addresses on xed-line broadband ISPs. Itis clear that devices “leave” individual IP addresses gradually,but at dierent rates on dierent ISPs. Aer one month,more than 90% of devices observed reconnecting to AT&T,RCN, and Cablevision do so on the same IP address. Aer

0 20 40 60 80Days Since Reference Observation

0.5

0.6

0.7

0.8

0.9

1.0

Frac

tion

of IP

s Mat

ched ATT

CablevisionCharterComcastRCNVerizonWOW

Figure 11: Persistence of IP addresses. e Figure shows theshare of night-time clusters on a single ISP and device, separatedby 𝑑 days, for which the IP addresses are equal on both clusters.Note that for visual clarity, the 𝑦 axis begins at 0.5 instead of0.

two months, more than three-quarters of devices return tothe same IP address, for all major ISPs in the three citiesshown.

6 CAN IP GEOLOCATION DATABASES BEUSED TO STUDY INTERNET ACCESS?

At this stage, we would usually turn to a general discussionof ndings. Here, we focus our discussion and extend ourresults, according to the question that originally motivatedour work: assessing the potential for using IP-referenceddata in social science research on Internet access. Where andfor what demographic groups is geolocation accurate? Can IPgeography enable Internet demography? To make this queryconcrete, imagine a study of the “homework gap” – (in)equityin access to digital resources for education – based solelyon server logs from a site like Wikipedia. If we observefrequencies of use by IP subnet alone, can we infer whatgroups do and do not access the site?General considerations. is question is non-trivial, sinceit confronts the correlations of population density and demo-graphics with geolocation accuracy, along with the spatialpaerns of connectionmodality (mobile vs xed). Cities havesmaller subnets simply because they have higher density ofpeople and devices. ey also tend to have larger minor-ity populations. is alone leads to a correlation betweengeolocation accuracy with demographics or disadvantage.For Chicago and its buer, the correlation between tract me-dian geolocation error on MaxMind (free) and populationdensity is −0.09 (𝑝 < 0.0001); in turn, population density iscorrelated with log median household income (𝑟 = −0.18,𝑝 < 10−10). Both of these are small but signicant. e ipside of beer accuracy at higher density is that distance pre-cision has to improve in dense environments, to associateactivity with the right population. It’s easier to “jump” overmany people when they are close together.

11

Accuracy also varies within the city, due to heterogene-ity in the fraction of people on mobile vs xed broadband.ere are two reasons for this. People use mobile devices(1) when they are on the go, or (2) because they do not haveaccess to a xed broadband connection at home. at meansthat devices in the present sample observed in city centersappear to have “inaccurate” IP geolocation, simply becausethe device users are more-likely on mobile on the way toor at work. On the other hand, populations without xedbroadband access are unlikely to be accurately IP geolocated,even in their home neighborhood.

As a nal consideration before proceeding, one must notconfound “unknown” addresses with “mis-located” ones. Forexample, if a default database location for T-Mobile addressessits in a particular neighborhood, that neighborhood willappear to have “accurate” geolocation, even though the loca-tions are not known any beer than elsewhere. Performancewill appear to “degrade” radially, with distance from the de-fault location. Since the default locations are usually in ornear cities, that would (ceteris paribus) give a false impressionthat IP addresses in cities (or near the center of the UnitedStates, for instance) are accurately-located.Dierences in access modality by demographic group.Returning to the data, Figure 12 presents the proportion ofthe night-time clusters in each tract of Chicago, that are onxed andmobile broadband. Note that the data are inherentlymobile devices with GPS chips; this does not include laptops,for instance. is classies AT&T, Comcast, WOW, and RCN,as xed-line providers, and T-Mobile, Sprint, Verizon, andAT&TMobile as mobile. For those familiar with Chicago, theresults are no surprise: the proportion of night-time pingson mobile networks is lower on the wealthier North Side ofthe city than on the West or South Sides. Indeed, our eyesdo not deceive us: the tract level correlation between thisconstructed variable and share of households with a broad-band contract as reported to the Census is −0.25. e cor-relation to neighborhood proportion Hispanic is 0.23 (both𝑝 < 10−10). In other words, connection type is correlatedwith demographic factors and broadband adoption. iswould be reected in geolocation accuracy. In practice, thismeans that constraining an analysis to accurately-located,xed-line IP addresses would tend to exclude vulnerablepopulations from the analysis.e inuences of density, demographics, and modal-ity on IP geolocation accuracy. Figure 13 oers an alter-native view of this eect, disentangling the countervailingforces of density, demographics, and access modality. Itdisplays the CDF of device geolocation accuracy for hyper-segregated neighborhoods of Chicago – ones where two-thirds of residents are white (only), Black (alone or in combi-nation with other races), or Hispanic (of any race). Moving

0.5 0.6 0.7Prop. of Night Clusters

on Mobile Networks

Figure 12: Proportion of night-time clusters in Chicagorecorded on mobile networks.

from le to right, we begin from the full dataset and layerthe cumulative requirements of devices in Chicago proper(not the 40 mile buer), at night (that is, likely at home), andon Comcast (i.e., on a single, xed broadband network). erst plot shows an enormous dierence between geolocationin “white” tracts and other segregated tracts – geolocationperforms much worse. is eect appears to have more todo with density than race: it reverses when focusing on theCity of Chicago, and zeroing in on a single network, theperformance lines up quite closely. e exception is at thevery high end (above 10 km and 90% of the CDF), wherethere is apparently an error for a set of white tracts. About80% of points are within 5 km of the true location, for allthree categories of neighborhood.Attenuation bias, from reliance on mis-attributed IPaddresses. e analyses of device modalities above sug-gest that IP geolocation databases’ ability to aribute onlinebehaviors to populations will tend to fail more oen fordisadvantaged groups. Still, if we were to persist, what er-rors might we expect to “accrue,” by moving an observationfrom its GPS-based location to the IP-based location? Inessence, this question pits the scale of geolocation accuracyagainst the physical scale of demographic segregation. If IPgeolocation moves a point among communities with similardemographics, the error does not directly bias results.

is analysis is limited to xed-broadband data from Com-cast, where geolocation has a chance of succeeding. Figure 14presents the log median household income as it would beimputed from a MaxMind look-up, against the true medianhousehold income of the neighborhood (Census tract). is

12

100 101

Geolocation Error [km]0.0

0.2

0.4

0.6

0.8

1.0

Cum

ulat

ive

Dist

ribut

ion

All Data

100 101

Geolocation Error [km]

Only Chicago...

100 101

Geolocation Error [km]

at night...

100 101

Geolocation Error [km]

on Comcast

BlackWhiteHispanic

Figure 13: Cumulative distribution of geolocation error for tracts with white, Black, and Hispanic super-majorities. e rst panelpresents all data, while the second through fourth restrict to Chicago, Chicago at night, and Chicago at night on Comcast.

10 11 12Neighborhood Log Median Household Income

10.0

10.5

11.0

11.5

12.0

Log

MHI

Impu

ted

from

Geo

loca

tion

Mediany = xQuartilesOuter Deciles

Figure 14: antiles of neighborhood log median householdincome as “imputed” from MaxMind geolocation (𝑦) as a func-tion of the true neighborhood value (𝑥).

results in an unsurprising regression to the mean: as is theusual case with measurement error, the slope is simply aen-uated. is suggests that even for xed broadband, eortsto use IP address alone to “link” online behaviors with hu-man populations are inadvisable at this physical scale. eywill in general yield estimates whose magnitudes are biaseddown. In other words, measurements of “who uses what”that rely on IP geolocation will tend to understate dieren-tial access. is is consistent with the ndings of Ganelinand Chuang on MOOC registrants and their geolocated IPaddresses. [7]

7 CONCLUSIONUsing a large sample of GPS-based smartphone locationsthis paper has quantied the performance of commercialgeolocation databases in large American cities. e preci-sion of this analysis far outstrips past work. e analysis hasdemonstrated signicant heterogeneity in geolocation accu-racy. e median error for MaxMind’s free service is wellless than 10 km on xed commercial broadband networksand at Universities. On mobile networks, IP geolocation isnot accurate below the city level.is analysis has also sought to explain why some ad-

dresses are accurately located whereas others are not. ephysical size of subnets is strongly correlated with accuracy.Large subnets appear to be used for “ephemeral” addressassignment, which clients do not use repeatedly.

Finally, we have contextualized these ndings for applica-tions to research on human populations. Both the presentdata and existing surveys show that disadvantaged popula-tions are less likely to use a xed broadband subscription athome. Online behaviors cannot be accurately associated withthese groups, and dropping mobile devices altogether willtend to remove them from analyses. Focussing on the con-text where geolocation does work – xed-line broadband –the accuracy still appears inadequate for associating onlineactivities with real-world geographies and demographics.From a privacy perspective, a single IP address does notidentify an individual, but it both localizes and individualand provides an “index” through time that may be used toaggregate other indirect identiers. We have shown that thetime for IP reassignment of xed-line broadband consumersvaries by ISP, but is typically on the order of months.

13

REFERENCES[1] Mohammed Jubaer Arif, Shanika Karunasekera, Santosh Kulkarni,

Ajit Gunatilaka, and Branko Ristic. 2010. Internet Host Geoloca-tion Using Maximum Likelihood Estimation Technique. In 24th IEEEInternational Conference on Advanced Information Networking andApplications. IEEE, Perth, Australia, 422–429.

[2] Ovidiu Dan, Vaibhav Parikh, and Brian D. Davison. 2018. IP Geoloca-tion through Reverse DNS. CoRR abs/1811.04288 (2018), 1–10.

[3] Brian Eriksson, Paul Barford, Bruce Maggs, and Robert Nowak. 2012.Posit: A Lightweight Approach for IP Geolocation. SIGMETRICSPerform. Eval. Rev. 40, 2 (October 2012), 2–11. hps://doi.org/10.1145/2381056.2381058

[4] Brian Eriksson, Paul Barford, Joel Sommers, and Robert Nowak. 2010.A Learning-Based Approach for IP Geolocation. In Passive and ActiveMeasurement, Arvind Krishnamurthy and Bernhard Planer (Eds.).Springer Berlin Heidelberg, Berlin, Heidelberg, 171–180.

[5] Dima Feldman, Yuval Shavi, and Noa Zilberman. 2012. A structuralapproach for PoP geo-location. Computer Networks 56, 3 (2012), 1029– 1040. hps://doi.org/10.1016/j.comnet.2011.10.029

[6] Michael J. Freedman, Mythili Vutukuru, Nick Feamster, and HariBalakrishnan. 2005. Geographic Locality of IP Prexes. In 5th ACMSIGCOMM Conference on Internet Measurement (IMC ’05). USENIXAssociation, USA, 153–158. hps://doi.org/10.5555/1251086.1251099

[7] Daniela Ganelin and Isaac Chuang. 2019. IP Geolocation Underesti-mates Regressive Economic Paerns in MOOC Usage. In 11th Inter-national Conference on Education Technology and Computers. Associa-tion for Computing Machinery, New York City, New York, 268–272.hps://doi.org/10.1145/3369255.3369301

[8] Manaf Gharaibeh, Anant Shah, Bradley Huaker, Han Zhang, RoyaEnsa, and Christos Papadopoulos. 2017. A Look at Router Geoloca-tion in Public and Commercial Databases. In Internet MeasurementConference (IMC ’17). Association for Computing Machinery, NewYork, NY, USA, 463–469. hps://doi.org/10.1145/3131365.3131380

[9] Manaf Gharaibeh, Han Zhang, Christos Papadopoulos, and John Hei-demann. 2016. Assessing co-locality of IP blocks. In IEEE Conferenceon Computer Communications Workshops. IEEE, San Francisco, CA,USA, 503–508. hps://doi.org/10.1109/INFCOMW.2016.7562129

[10] Phillipa Gill, Yashar Ganjali, BernardWong, and David Lie. 2010. Dude,Where’s at IP? Circumventing Measurement-Based IP Geolocation.In 19th USENIX Conference on Security (USENIX Security’10). USENIXAssociation, USA, 16. hps://doi.org/10.5555/1929820.1929842

[11] Bamba Gueye, Steve Uhlig, and Serge Fdida. 2007. Investigating theImprecision of IP Block-Based Geolocation. In Passive and Active Net-workMeasurement, Steve Uhlig, Konstantina Papagiannaki, and OlivierBonaventure (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg,237–240.

[12] Bamba Gueye, Artur Ziviani, Mark Crovella, and Serge Fdida. 2006.Constraint-Based Geolocation of Internet Hosts. IEEE/ACM Trans.Netw. 14, 6 (Dec. 2006), 1219–1232. hps://doi.org/10.1109/TNET.2006.886332

[13] Zi Hu, John Heidemann, and Yuri Pradkin. 2012. Towards Geolocationof Millions of IP Addresses. In Internet Measurement Conference (IMC’12). Association for Computing Machinery, New York, NY, USA, 123–130. hps://doi.org/10.1145/2398776.2398790

[14] Bradley Huaker, Marina Fomenkov, and kc clay. 2011. Geocom-pare: a comparison of public and commercial geolocation databases.Technical Report. Cooperative Association for Internet Data Analysis(CAIDA), San Diego, CA. hps://www.caida.org/publications/papers/2011/geocompare-tr/

[15] Ethan Katz-Basse, John P. John, Arvind Krishnamurthy, DavidWetherall, omas Anderson, and Yatin Chawathe. 2006. Towards

IP Geolocation Using Delay and Topology Measurements. In 6thACM SIGCOMM Conference on Internet Measurement (IMC ’06). As-sociation for Computing Machinery, New York, NY, USA, 71–84.hps://doi.org/10.1145/1177080.1177090

[16] Dan Komosny and Saeed Ur Rehman. 2020. Survival Analysis andPrediction Model of IP Address Assignment Duration. IEEE Access 8(2020), 162507–162515. hps://doi.org/10.1109/ACCESS.2020.3021760

[17] Measurement Lab. 2021. M-Lab Visualizations. (April 2021). hps://www.measurementlab.net/visualizations/

[18] David Lazer, Alex Pentland, Lada Adamic, Sinan Aral, Albert-LaszloBarabasi, Devon Brewer, Nicholas Christakis, Noshir Contractor,James Fowler, Myron Gutmann, Tony Jebara, Gary King, MichaelMacy, Deb Roy, and Marshall Van Alstyne. 2009. Computational So-cial Science. Science 323, 5915 (2009), 721–723. hps://doi.org/10.1126/science.1167742

[19] Dan Li, Jiong Chen, Chuanxiong Guo, Yunxin Liu, Jinyu Zhang, ZhiliZhang, and Yongguang Zhang. 2013. IP-Geolocation Mapping forModerately Connected Internet Regions. IEEE Transactions on Paralleland Distributed Systems 24, 2 (2013), 381–391. hps://doi.org/10.1109/TPDS.2012.136

[20] Vikas Mishra, Pierre Laperdrix, Antoine Vastel, Walter Rudametkin,Romain Rouvoy, and Martin Lopatka. 2020. Don’t Count Me Out: Onthe Relevance of IP Address in the Tracking Ecosystem. In Proceedingsof e Web Conference 2020 (WWW ’20). Association for ComputingMachinery, New York, NY, USA, 808–815. hps://doi.org/10.1145/3366423.3380161

[21] Venkata N. Padmanabhan and Lakshminarayanan Subramanian. 2001.An Investigation of Geographic Mapping Techniques for InternetHosts. In Conference on Applications, Technologies, Architectures, andProtocols for Computer Communications (SIGCOMM ’01). Associationfor Computing Machinery, New York, NY, USA, 173–185. hps://doi.org/10.1145/383059.383073

[22] Mayana Pereira, Allen Kim, Joshua Allen, Kevin White, Juan LavistaFerres, and Rahul Dodhia. 2021. U.S. Broadband Coverage Data Set: ADierentially Private Data Release. (April 2021), 1–7.

[23] Ingmar Poese, Steve Uhlig, Mohamed Ali Kaafar, Benoit Donnet, andBamba Gueye. 2011. IP Geolocation Databases: Unreliable? SIGCOMMComput. Commun. Rev. 41, 2 (April 2011), 53–56. hps://doi.org/10.1145/1971162.1971171

[24] Yuval Shavi and Noa Zilberman. 2011. A Geolocation DatabasesStudy. IEEE Journal on Selected Areas in Communications 29, 10 (2011),2044–2056. hps://doi.org/10.1109/JSAC.2011.111214

[25] Neil Spring, Ratul Mahajan, and David Wetherall. 2002. MeasuringISP Topologies with Rocketfuel. In Conference on Applications, Tech-nologies, Architectures, and Protocols for Computer Communications(SIGCOMM ’02). Association for Computing Machinery, New York,NY, USA, 133–145. hps://doi.org/10.1145/633025.633039

[26] Yong Wang, Daniel Burgener, Marcel Flores, Aleksandar Kuzmanovic,and Cheng Huang. 2011. Towards Street-Level Client-Independent IPGeolocation. In 8th USENIX Conference on Networked Systems Designand Implementation (NSDI’11). USA, 365–379. hps://doi.org/10.5555/1972457.1972494

[27] Bernard Wong and Ivan Stoyanov. 2007. Octant: A Compre-hensive Framework for the Geolocalization of Internet Hosts.In 4th USENIX Symposium on Networked Systems Design &Implementation (NSDI 07). USENIX Association, Cambridge,MA, 313–326. hps://www.usenix.org/conference/nsdi-07/octant-comprehensive-framework-geolocalization-internet-hosts

[28] Inja Youn, Brian L. Mark, and Dana Richards. 2009. Statistical Geolo-cation of Internet Hosts. In 18th International Conference on ComputerCommunications and Networks. IEEE, San Francisco, CA, USA, 1–6.hps://doi.org/10.1109/ICCCN.2009.5235373

14

A ADDITIONAL PLOTS AND TABLES

Cluster Class Bumps ClustersLong Area Dwell 0.382 0.079Travel 0.322 0.264Area Dwell 0.193 0.173Short Area Dwell 0.074 0.290Potential Area Dwell 0.025 0.127Ping 0.003 0.066Large Variance 0.001 0.000Moving 0.000 0.000Split 0.000 0.000

Table A.1: Proportion of bumps and clusters according to theclassication type assigned by Unacast (cf Section 3.1).

NIC Frac.AFRINIC 0.00011APNIC 0.00015LACNIC 0.00016RIPE 0.00169

Table A.2: Proportion of clusters with IP addresses in foreignInternet registries (cf Section 3.1).

100 101 102 103

Geolocation Error [km]0.0

0.2

0.4

0.6

0.8

1.0

Cum

ulat

ive

Dist

ribut

ion

MaxMindIP2Location

Not TravelTravel

Figure A.1: Empirical cumulative distribution of geolocationaccuracy, for travel and non-travel clusters on AT&T’s mobilenetwork, as evaluated on the free versions of the MaxMind andIP2Location databases (cf Section 4.1).

15