UK open public data

Ordnance Survey, the British national mapping agency, has released open data at the beginning of this month. This is a 180 degree change for Britain. It used to have one of the most restrictive public data policies and hardly any official geo data was available. The new British open data policy is one of the most liberal policies worldwide and we can hope that other countries, in particular in Europe, will also understand the importance of unrestricted access to public data and adopt similar open policies.

The data is not only called ‘open’ it really is open. Unlike other ‘open data’ projects that use unopen share-alike licenses,  the OS OpenData is available under a so called ‘OS OpenData License‘, which is aligned and interoperable with the creative commons attribution license.

Of interest for GeoNames are several datasets:

• 1.7 million postal codes (code-point)
• 1:50’000 Gazetteer (260’000 toponyms)
• admin boundaries

It will take some time and work till we can make optimal use of the data. The existing ‘outcode’ based postal codes have already been replaced with the new data. On the free ws.geonames.org server the postal code webservices (reverse and search) are now returning the full 1.6 million postalcodes. The dataset does not include postal codes for NIR, IM, GY and JE. For those regions we continue using the previous data.

The postal code dump directory now contains two files for GB, the default file with the outcodes and an additional file with the full postal codes. It is not yet clear whether we should continue with the outcodes or replace them entirely with the full codes. What do you think? Please comment your ideas and requirements below.

In order to use the admin boundaries we will have to clean up the existing admin divisions and align them with the Ordnance Survey divisions.

The gazetteer unfortunately only has a very high level ‘feature code’ and for the majority of the toponyms the feature code is missing entirely (X).

fcode | count
-------+--------
X | 128662 (all other features)
O | 41228 (other)
FM | 34723 (Farm)
W | 24425 (Water)
H | 14524 (Hill or mountain)
F | 8708 (Forest or wood)
A | 5252 (antiquity non-roman)
T | 1259 (town)
R | 237 (roman antiquity)
C | 62 (city)


DDOS part II

The free web services are timing out at the hour precisely for a short moment of some seconds. The reason is a widget that calls the services timezoneJSON and  findNearByWeatherJSON always at exactly the full hour from a large number of ip addresses. The sudden spike in requests is causing many other requests to timeout. Around a year ago the free services were suffering from the effects of an iphone application that has become very popular and was using some geonames web services.

Some hours ago we have changed the service to throw an exception hoping that the developer of the widget will see that the application no longer works and change the behavior of the application. It is not very useful if a distributed application running on a huge number of clients is calling the same server at the very same instance.

The exception is thrown on the domain ws.geonames.org for requests of the two JSON services and if no parameter username is present. If you happen to be using the service, just add the parameter username=<your geonames username> avoid the exception. Those using a ‘secret’ domain name are not affected. You can create an account here.

‘XK’ country code for Kosovo

Some of you have already noticed that we are now using ”XK‘  as temporary country code for Kosovo. While the US standards body ‘FIPS’  has found it worthwhile to assign a country code to Kosovo (KV), the International Organization for Standardization, ISO, has yet to assign a code to the former Serbian province. The ISO country code standard 3166 has a couple of unused codes that can be used for user specific elements: “If users need code elements to represent country names not included in this part of ISO 3166, the series of letters AA, QM to QZ, XA to XZ, and ZZ, and the series AAA to AAZ, QMA to QZZ, XAA to XZZ, and ZZA to ZZZ respectively and the series of numbers 900 to 999 are available.“

The European Commission and many other organisations (Deutsche Bundesbank, Switzerland) are using ‘XK‘ as a temporary country code for Kosovo till ISO officially assigns a code.

GeoNames will switch to the official ISO code as soon as it has been released. In the meantime we will use ‘XK‘.

Links for Toponyms

A new pseudo language code ‘link‘ has been added to the alternate name edit function and the links to the English Wikipedia have been inserted as alternate names. The links to the corresponding wikipedia articles have often been requested. While they were available on the forum linked in some threads, they were not included in the normal dump. With this simple change they can now be included in the dump as alternate names and they can easily be maintained using the wiki interface. All other kind of links, I think of hotel websites for hotel entries, can also be added in the same manner.

The language code for the alternate names are normally the 2-character ISO 639 language codes, for more exotic languages that do not have a 2-character ISO code the 3-character code is used instead.

Pseudo codes

• ‘post‘ for postal codes
• ‘link‘ for a link to a website
• ‘iata‘, ‘icao‘ and ‘faac‘ for the respective airport codes
• ‘abbr‘ for an abbreviation
• ‘fr_1793‘ for names used during the French Revolution

GeoNames Logo

Some weeks ago GeoNames got a fantastic new index page design to replace the oversimplistic previous index page. The new index page does not only look nicer it also makes it easier for visitors to quickly get to the information they are looking for. Many thanks to Erik Bolstad for this great contribution.

GeoNames Homepage Mai 2009

Today I would like to ask you for your preference for a GeoNames logo. GeoNames did not have a real logo for many years, till Alexander Torrenegra and his team from LetMeGo finally stepped in and designed some cool logo ideas.

What to do against DDOS effects?

A week ago we had to take down the subdomain ws.geonames.org that we use for the free web services. The server was flooded with requests from iMob an iPhone application that has gone viral and become one of the most popular iPhone applications. It is currently number one in the free games section. There were too many connections attempts that blocking the requests by their user agent did not help and we had to disable the domain completely to get the service to work on an alternate sub domain.

To avoid a complete knockout by a single application we have now defined a list of alternate subdomain, that we are not going to publish to make sure an offending application is not using all subdomains and bringing down all application using the free services.

Drop me an email if you want to get one subdomain name you could use for your application.

IPhone applications are particularly nasty as they are not coming from a single IP address, it takes some time to get a new release approved by Apple and moved into the App Store and last but not least it takes time for all users to upgrade to the newest release.

Edit 10. March, Remark: The subdomains are using the same physical hardware. For better response time and higher availability use the commercial services.

Twitter GeoVisualisation

Walter Rafelsberger from MODUL/University Vienna/Department of New Media Technology is using GeoNames for some interesting and beautyful geo visualisation projects.

Twitter Conversations

The twitter conversation map shows where people talking to each other are located.

Twitter Weather Map

For the Twitter Weather Map the location and weather information twitter users are posting is parsed and visualised:

GeoNames cities over 1000/5mio

GeoNames cities over 1000

GeoNames web service client r1.0

The GeoNames web service client for java has been released in version 1.0. The release includes the following changes and additions :

• implemented children and neighbours service
• throw exception if a field is accessed that has not been set due to insufficient style parameter
• add adminname1 and adminname2 to Toponym
• fixed a couple of minor bugs
• added and improved documentation
• support for username and token for authentication
• client failover to alternative server if the main server is not accessible. The failover server will be used for some minutes before the client will automatically try to switch back to the main server. This is a simple and efficient approach to achieve high availability for an application.

Linked Data and the Semantic Web

The Semantic Web, the web of data, is coming of age and making it recently into main stream news coverage. GeoNames was among the first to offer a geographical ontology and RDF web services and GeoNames is also part of the Linked Data project. The Linked Data project brings together data from public sources and builds a web of open and free data where data sets are interlinked with each other. Geographic concepts are referred to using the GeoNames URI with the unique GeoNames Identifier the geoNameId.

[Image : Projects involved in the Linked Data project (Feb 2008, Richard Cyganiak)]

Tim Berners-Lee has written an interesting blog posting about how a misquote from the Times interview spread over the web and could not be stopped. (He also mentions GeoNames.)

GeoTree – Hierarchical Toponym Browser

GeoTree is a new hierarchical toponym browser for GeoNames. It allows to drill down the continents and the administrative divisions of a country in an explorer like fashion. To the right of the tree view a map shows the toponym selected. An outstanding component of GeoTree are the flags and coat of arms displayed with most administrative features. The coat of arms are from the wikipedia Blasons (heraldry) project. Moving the mouse over the name of an administrative division will not only focus the map, it will also display a larger version of the coat of arms.

The GeoNames balloons are linking to the respective GeoTree representation of the toponym. GeoTree is using the GeoNames hierarchical webservices.

GeoTree is developed by Christophe, GeoNames Ambassador to France. Jan and Bernard are helping with the svg representation of the coat of arms. Check it out.