I had the requirement of extracting hospitals from OpenStreetMap into Postgresql so that some data analysts could run some scripts off it.

I fired up an AWS EC2 server (t3.large) running Ubuntu and installed the awscli package and configured it using my AWS Key and Secret Key.

sudo apt-get install awscli
aws configure

Then I downloaded the PBF file from AWS S3 (you don't incur a charge if downloaded from S3 if you are using EC2)

aws s3 cp s3://osm-pds/2018/planet-180903.osm.pbf .

The file size was about 42GB and took 20 minutes to download. Next, I used Imposm3 instead of Osm2pgsql to read the PBF file and import into postgresql server.

imposm import -config config.json -appendcache -read planet-180903.osm.pbf -write

The config.json specified the database and mapping parameters.

{
     "cachedir": "/home/ubuntu/cache",
     "connection": "postgis://osm:osm@localhost:5432/osm",
     "mapping": "mapping.json",
     "srid": 4326
}

Since I was only interested in the location of hospitals and didn't require other tags or admin boundaries, relations etc, this is how my mapping.json looked.

{
     "tags": {"load_all": true},
     "tables": {
         "amenities": {
             "columns": [
                 {
                     "type": "id",
                     "name": "osm_id",
                     "key": null
                 },
                 {
                     "type": "geometry",
                     "name": "geometry",
                     "key": null
                 },
                 {
                     "type": "string",
                     "name": "name",
                     "key": "name"
                 },
                 {
                     "type": "mapping_value",
                     "name": "type",
                     "key": null
                 },
                 {
                     "type": "hstore_tags",
                     "name": "tags",
                     "key": "tags"
                 }
             ],
             "type": "polygon",
             "mapping": {
                 "amenity": [
                     "hospital"
                 ]
             }
         }
     }
 }

The imposm import command took nearly 19 hours to import the data into postgresql. You can push the data into the public schema using the -deploytoproduction flag or just use the import schema if you are comfortable with it.

Additionally, I exported the data from the osm_amenities table into a json file (a json line for every row in the table) using the following PSQL command:

COPY (SELECT row_to_json(t)
      FROM (SELECT id, osm_id, name, type,
                   hstore_to_json(tags) as tags,
                   ST_AsGeoJSON(geometry)::json as geometry
            FROM import.osm_amenities) t
     )
TO '/tmp/output.json';

My Muttrc

Thu 13 September 2018 by Thejaswi Puthraya

I have been using Mutt, a text-based email client for ages for some self-hosted email that I use internally. Someone recently asked me to share my .muttrc and so here goes:

set editor = "emacs -nw"

set mbox_type=Maildir
set mbox="~/Mail/Inbox"
set spoolfile="~/Mail/Inbox"
set folder="~/Mail/"
set …
read more

A People's History of the United States

Fri 31 August 2018 by Thejaswi Puthraya

"History is written by the victors" goes a famous quote. Every reading of history should be treated with some scepticism. "A People's History of the United States" written by Howard Zinn narrates history not through the government's propaganda but through common people's movements.

The book opens with Christopher Columbus' armada …

read more

Port Forwarding with HAProxy

Sat 11 August 2018 by Thejaswi Puthraya

I recently acquired a beefy bare-metal server and wanted to run a bunch of services within VMs based on KVM managed by libvirt. Only ports 80 and 443 for these VMs would be exposed and the rest of the ports (say SSH) visible only from the internal network.

Initially, I …

read more

Shantaram

Sat 28 July 2018 by Thejaswi Puthraya

Looks can be deceptive. Very deceptive. When some villagers name the protagonist "Shantaram" after his calm nature, little idea did they have of his fierce past. Shantaram tries hard to melt into his surroundings and maintain a low-key but keeps getting into trouble.

Gregory David Roberts, the author, embellishes his …

read more

When breath becomes air

Mon 11 June 2018 by Thejaswi Puthraya

What would you do if you are informed that you are dying (no timeline provided)? Would you continue working towards your dream? Or give up and spend the rest of the time cursing your fate?

Paul Kalanithi is close to completing his neurosurgeon residency at Stanford. He has worked hard …

read more

Mahashweta

Fri 08 June 2018 by Thejaswi Puthraya

Dr.Anand falls in love with Anupama and it appears to be progressing like a fairy tale. After marriage, her husband leaves to England for higher studies and she is expected to follow him shortly. But she is diagnosed with Vitiligo and is shunted out of her in-laws and parents …

read more

On the Origin of Species

Fri 01 June 2018 by Thejaswi Puthraya

I have heard of the phrase "Survival of the fittest" since childhood in contexts unrelated to Charles Darwin's (this phrase was added in a later edition) usage. It always piqued my curiousity on how he (and some people before him) proposed this revolutionary idea. They didn't observe the evolution of …

read more

Pakistan; or the partition of India

Thu 29 March 2018 by Thejaswi Puthraya

Every person in the Indian subcontinent is aware of the misery caused by the partition of India. A lot of people believe that the Indian Congress Party and the first Prime Minister of India were responsible. They insist that the partition could have been avoided and history would've been kinder …

read more

The Bourne Identity

Tue 13 March 2018 by Thejaswi Puthraya

Countless number of times people have recommended this book but I pushed back by claiming that this book didn't fit my interests. Recently I realised that my reading has been wide-ranging and I don't have a favourite genre.

A man is found almost dead by fishermen in the middle of …

read more