What to do against DDOS effects?

A week ago we had to take down the subdomain ws.geonames.org that we use for the free web services. The server was flooded with requests from iMob an iPhone application that has gone viral and become one of the most popular iPhone applications. It is currently number one in the free games section. There were too many connections attempts that blocking the requests by their user agent did not help and we had to disable the domain completely to get the service to work on an alternate sub domain.

To avoid a complete knockout by a single application we have now defined a list of alternate subdomain, that we are not going to publish to make sure an offending application is not using all subdomains and bringing down all application using the free services.

Drop me an email if you want to get one subdomain name you could use for your application.

IPhone applications are particularly nasty as they are not coming from a single IP address, it takes some time to get a new release approved by Apple and moved into the App Store and last but not least it takes time for all users to upgrade to the newest release.

Edit 10. March, Remark: The subdomains are using the same physical hardware. For better response time and higher availability use the commercial services.

Advertisement

24 thoughts on “What to do against DDOS effects?

  1. Geonames city database fits easily into a SQLite database embedded inside an iPhone app. Such an approach is fast for the user and puts no load on your servers. Feel free to send iPhone developers my way for tips how to do this. If there is enough interest perhaps I will open source the solution.

  2. Hi Mark –

    I’d be interested in hearing the tips you have for embedding geonames data in a SQL Lite DB. I am an experienced developer that is new new to iPhone development. The first app I am experimenting with is a location-aware application that can make good use of the data that geonames provides. I am currently using the geonames webservices to grab the data, and have already noticed that the free service uptime has been spotty lately. Storing the data locally will of course address this, as well as improve repsonse time (at the expense of data ‘freshness’). I’d appreciate any info re: geonames data + SQL Lite that you be willing to pass along. Thx.

    Joe

  3. I’m currently working on a geonames-based app for an unnamed platform, and one thing that i’ve been doing is using a LRU-cache for data. I figure that there’s nothing in geonames that would be changing fast enough to cause issues with this.

    i’d be happy to write a blog post concerning this on my own blog concerning this, but in general it seems that there are a number of simple best practices that you should encourage/enforce on people to minimize your load.

    (i’m also planning to release my caching lib as open source once i’m done.)

  4. Hi Marc and Geonames,

    I would like to say thank you for your work. I have implemented your 1000cities table in a database to find towns near a dive site. The website is still in development but I wanted to say thanks for making this data available.

    Phil

  5. Hi, Marc. Thanks so much for the db and the web services. I only need to use the service sparingly for now (still in development) but I can’t find your email address to ask for an alternate subdomain to query.

    Again, thanks very much,
    Sean Anderson

  6. Hi,
    I have used the web services about once or twice a year to assist in planning hiking trips that our group goes on regularly. I have software that combines google maps and altitudes from geonames to give a good indication of what we’re up against, and allow good planning.
    Unfortunately, it is very difficult for me to get even 1 altitude at the moment, and these take upwards of 3 minutes each.
    As I don’t have your email, a secondary sub-domain would be very much appreciated.
    Regards,
    Hamish.

  7. Hi Marc,

    I can’t find your email on the blog ?
    Please could you contact me ? A sub-domain would be appreciated for my website, I just need WS geonames a few times a day but if this service doesn’t work I get some errors …
    Regarding my number of visits a day, I will not break your WS 🙂

    regards,

    Stéphane

    • Hi Stéphane

      I have sent you an email. The alternate sub-domain are sharing the same hardware, the advantage is that we can disable subdomains selectively in case of massive DDOS by a single application. The problem we had on Tuesday would not have been solved with the subdomain. (A server was causing problems and there are too many request that the other servers can cope with the additional load)

      Marc

  8. Hello Marc,

    I have to say thanks for this great service for free, I just currently using to get the time for a specified latitude and longitude, I would like to know at least for additional servers, on which I can rely to query when some of those fails to resolved my request.

    Thanks, José Peña

  9. Hi Jose

    I cannot pay for an infinite number of servers. When there are too many requests from too many people there is nothing I can do. If you need high availability then you will have to go with the commercial offering and help share the cost for the servers.

    The blog posting is about the situation when a single application is causing troubles and shutting down one subdomain instead of shutting down the entire server.

    Best

    Marc

  10. Hi Marc

    I use the srtm3 to find elevations on some internal homepages, with less visitors. Can I have a link to a subdomain to use instead?

    Please send me an email.

    Thanks a lot for the service.

    Aage

  11. Hi Marc,

    I’m protyping a website and would like to pull minimal data from one your services during development. If the prototype is accepted we will definitely convert to the commercial services and help support the cause. Is there a sub-domain I can use while I prototype the initial development.

    Thanks,
    Brad

  12. Hello Marc,

    I am currently looking at options to validate address information in an existing app that I support. If we decide to use the geonames ws, then we will pay for the commercial version. I noticed today that http://ws.geonames.org appears to be working again. Is this the site that I should develop against? If not, could you send me sub-domain?

    Thanks

  13. Hi Marc – thanks for the info. We’re building an iPhone app as well and interested in your idea of loading the data locally on the iPhone. You referenced that the city database can easily fit within an app – are there other categories of content that would fit as well?

    Thanks

  14. I have a railsgem that does caching of geonames results locally and can be used to cut the load from applications way down. if anyone is interested in beta testing it, let me know. I am going to put it up on github in april, but am doing local tests to make sure that it doesn’t destroy the known universe until then.

    http://corprew.org/contact

  15. Hi Mark

    Thank You for the service. I had some problems with the subdomain suggested, but have now the possibility to use googles elevation services….
    Do they use the same data?

    Aage

  16. Mark, can you E-mail me a domain name? I’ve made an Android widget accessing your services, and giving you the possibility to track it specifically would be great.

    Widget is at launchpad.net/thermometer.

    Thanks for an awesome service!

    Cheers /J

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s