Even with the best of maps and instruments, we can never fully chart
– Gail Pool
The questions worth asking have not answers worth knowing, but
rather they engender journeys worth taking.
– Curt Brune
This article describes the motivations behind and the inner workings of a "IP address to Google Map"
mash-up application I wrote recently called Geo IP Route. Read on!.
What should I do?
Where should I go?
Am I sure this is the right direction?
What if I had chosen a different path?
Most of us have asked ourselves these types of questions at one
time or another. It's only natural to ponder the meaning of our
existence and wonder aloud at the complexity of the universe.
Like all good questions these questions apply to many different
levels of our existance, from superficial daily decisions to
subdermal compromises of family and career to the innermost
vexing decisions regarding ethics and morality.
Off hand the questions arising from routine daily decision
making are of little interest to me. "Don't sweat the small
stuff," someone once said. Good advice as life is too short.
Make decisions, take action and keep moving forward.
No, I find the subtle, difficult questions much more
intriguing. As these decisions lie closer to our core being,
the effects of minor course corrections dramatically
influence the route we travel. While it's true destinations
help focus your energy, upon arrival the journey gives more
insight to where you are then the destination itself.
I would like to share with you one of my recent existential
programming journeys. Along the way I ponder questions, fiddle
and geocoding. As
I've written previously
the best way to get going to is to have a project to work on.
With the project in hand we can sharpen the saw and get to work.
Here's a snapshot of the final application:
Read on for more ...
Charting Your Route
I wanted to do something new, something new to me and to you.
Also I wanted to do something different from my day job
(embedded system design and programming). For the something
different I wanted to learn how to make web pages with
interactive maps using the Google Maps API.
If you have been following along you know I have a history
of working with GIS and digital map applications. I had never
played with online map applications though.
Now the something new to me and you was going to be
difficult. Mapping APIs are great, but what are you going to
put on the map? To put items on a map you first need their
latitude and longitude, otherwise known as geocoding. Where
could I get geocodes of something interesting? For free
After much head scratching I decided it would be cool to map the
Internet routers used for various transactions. If you are
familiar with traceroute
you know exactly what I was thinking.
I wanted to make sure what I was doing didn't already
exist. I did a Google search for "traceroute geocoding" – much to
my dismay several sites came up. Only one seemed relevant, in
fact it was pretty much exactly what I was thinking of doing.
Somewhat long faced I clicked on the link to check out the
application. Much to my delight the application was
non-existant, the link provided didn't work. I was still in
This project had several challenges that had to be surmounted in
order to succeed.
The first trick was to convert the Internet
Protocol (IP) addresses into latitude and longitude
coordinates. I had a couple of ideas on how to do this.
The second trick was learning the Google Maps API and
glue it all together. While this seemed straightforward at
first plenty of surprises were lurking to insure an eventful journey.
With destination in hand, the navigation charts arrayed before
me I was ready to embark on the journey.
IPv4 to Latitude and Longitude
How to do this? Companies
sell databases that map IP addresses to latitude and
longitude, along with a subscription service to keep the
database up to date. That would likely be the mostly accurate
way to go, but that would take all the fun out of it. Plus I
don't have money to pay for it.
I harkened back to my experience which traditional geocoding,
where the latitude/longitude (lat/long) is derived from a street address.
Maybe a I could transform IP addresses to street addresses and
then to lat/long. How could I get street addresses from IP
Given that this was a for-fun-not-for-profit project 100%
accurracy was not required, so I decided to use the whois database to look up the
administrative contact for given IP addresses. This was not the
final solution I used for the finished application, but it was a
start. Starting is key to any journey!
Clearly using whois was not going to tell me the exact location
of the server in question, but it would tell me the location of
the organization that ran the server. Good enough.
The following is the abbreviated whois output for LinuxDevices.com. First I
looked up the IP address for www.linuxdevices.com and then fed
that IP address into the whois command, using the whois.arin.net whois sever.
linux:$ whois -h whois.arin.net + 18.104.22.168
OrgName: Hurricane Electric
Address: 760 Mission Court
Hurricane Electric is a co-lo in the San
Jose, CA area. This is likely where the servers for
LinuxDevices.com reside. Considering all the places the server
could have been, this narrows it down considerably.
OK, great so I have the street address given an IP address. Now
I need a geo-coder. Free
geocoders exist as web services, which are convienent, but
also introduce latency as the query is serviced by a third-party.
I didn't want to introduce more latency – I was already querying
the whois database.
I wanted to do the geo-coding locally on my own server. Luckly
the U.S. Census
Bureau makes its Tiger Line data freely available for download.
A nifty perl module, Geo::Coder::US,
provides an interface to this database.
I was now in business – I could turn an IP address into a
location on the map. While not optimal it got me going so I could
move on to the next challenge, using the mapping APIs and making
something worth looking at.
less than you when I embarked on this adventure. Sure I had
needed a good reference to get up to speed.
I found the W3
It is really, really helpful with nice working examples. Go
there and read up.
The Google Maps
API is well documented and pretty straight forward to use.
My applicatin required two basic mapping features
- Draw a map, centered on a lat/long at a particular zoom
- Put icons on the map at particular locations.
Most mapping applications do little more than this. The Google
APIs made this a relatively easy task.
In my application the user sends a series of IP addresses to my
server, where I turn those IP address in to lat/long locations
and send them back to the user's browser. The highlight for me
was the using XML to send the data back to the browser.
The Google API provides a really nice
XML parser making it simple to marshall arguments
between web applications. What a time saver! Totally cool.
functions for asynchronous HTTP query/response handling.
First an HTTP request is made and an anonymous function is
registered as the response handling function. Later the
response is handled asynchronously by the registered function.
It was cool using an anonymous function for this –
reminded me of the lambda
functions of LISP.
Iteration and Closure
We are what we repeatedly do. Excellence then is not an act but a
Perfection is achieved, not when there is nothing more to add,
but when there is nothing left to take away.
– Antoine de Saint Exupery (1900 - 1944)
Just like writing an essay, creating good computer programs
usually benefit from one or more development iterations –
quickly putting down the first draft captures the fleeting flashes
of insight bouncing around your ideas. Subsequent drafts smooth out the
rough spots, refine the algorithms, shorten the code paths
and delete the cruft.
Earlier I alluded that the final version of my application did
not stick with the whois/geocoding method for turning IP
addresses into lat/long coordinates. During development I found
a more satisfying method.
The first enhancement was to realize that someone else was
already offering a free service for U.S. and U.K IP addresses.
Go check out hostip.info.
This solved the problem for two large countries, but what about
the rest of the world?
Well that was going to be tricky. In the end I knew the
application did not need to be perfect, but only provide an
estimate for the path that packets took across the Internet.
The hostip.info query fails whenever it does not have a location
for an IP address – it can sometimes fail even for
U.S. and U.K. addresses that it does not know about. When it
fails, however, it does tell you the country that the IP address
If the country is the U.S. then I fall back on the my original
whois/geocode method, which works OK. For other countries I use
the lat/long of the capital city of the country. So the resuls
are not perfect, but the spirit of where the packets are going is
preserved. I used the list of world capital lat/longs found here.
Until Next Time
Well I hope you have enjoyed this journey – good travels!
Oh, I almost forgot – please check out my Geo IP Route application.
Posted by curt at December 18, 2005 2:48 PM