A quick guide to running your own tile server, with a geo database populated from OpenStreetMap data.

Your own map server

With this project, you will be able to:

• Launch your own GIS database and tile server
• Populate the database with data from OpenStreetMap (OSM)
• View a map generated by your tile server

The code is in a GitHub repository, and there’s also a quick guide there to getting started:

• instantiator/world-server

Technologies

As the purpose of this project is to simplify running and connecting a variety of existing tools, the choice of technologies was guided by simplicity of containerisation. We’re using:

• Docker - to run containers for individual applications.
• Lua - a simple scripting language.
• PostGIS - a postgres database with geographical capabilities.
• osm2pgsql - a tool to import OSM data into a PostGIS database.
• pg_tileserv - a tile server with a simple map interface

Data

Data comes from OpenStreetMap (OSM). GEOFABRIK host mirrors of OSM data as downloads, broken down by region. The data is provided as pbf files (Protocolbuffer binary format).

For this project, we’ll use data from Great Britain.

Getting started

First, clone the project code to your machine:

$ git clone https://github.com/instantiator/world-server.git

Downloading the data

The update-data.sh script is a simple way to fetch the data from GEOFABRIK.

If you’re going to fetch an area other than Great Britain, you’ll want to modify these variables in the script:

DATA_PATH=europe
DATA_FILE=great-britain-lastest.osm.pbf
BACKUP_FILE=great-britain-lastest.osm.pbf.backup

DATA_PATH and DATA_FILE are composed into a GEOFABRIK url. Run the script to download your chosen data into the data/ directory:

$ ./update-data.sh

Launching the servers

compose.yaml defines a Docker Compose application that combines and connects:

• world-server-db (PostGIS) - the database
• tile-server (pg_tileserv) - a simple tile server

There are a number of settings worth noting:

• world-server-db (PostGIS) has a healthcheck that runs when it starts up.
• tile-server has a dependency on world-server-db and so will wait until the database is ready to start.

world-server-db has a volume called db-data-world-server that points to /var/lib/postgresql/data/ in the container. This volume effectively preserves the database contents across container states.

There are a number of variables in config/world-server.env that control these components, too:

• tile-server is exposed on the port specified in TILE_PORT.
• world-server-db is exposed on the port specified in DB_PORT.

You shouldn’t need to change much, but you may wish to change the password in a production environment (the default password provided is not a secret and should be treated with caution).

Ensure that Docker is running on your system, and launch the application:

$ ./run-server.sh

Import the data

Use the import-data.sh script to import the data you previously downloaded into your PostGIS database, using osm2pgsql.

By default it’ll import data/great-britain-latest.osm.pbf, using scripts/import-campsites.lua to filter the data.

Configure the import with these parameters:

Import data from a provided .osm.pbf file into a local postgis database.

Options:
    -d <path>     --data <path>       Specify the .osm.pbf data source
    -s            --script <path>     Control import activity with a lua script
    -a            --all               Import all data from the data source
    -h            --help              Prints this help message and exits

Defaults:
    -a false
    -d data/great-britain-latest.osm.pbf
    -s scripts/import-campsites.lua

NB. If you specify the --all parameter, no filtering script will be used, and the entire dataset will be imported. This can take a long time!

To use the defaults, which will import campsite data from Great Britain, run the script with no options:

$ ./import-data.sh

Import filtering

osm2pgsql supports the use of a Lua script to control the import - effectively configuring the database table to import to, and providing functions to filter the import.

See: The Flex Output

By default, the import script uses scripts/import-campsites.lua - a Lua script that looks for campsite here. Use the --script option to provide another. There are plenty of lua examples at: https://github.com/openstreetmap/osm2pgsql/tree/master/flex-config

osm2pgsql provides some functions to support the import:

• osm2pgsql.define_table is used to define tables
  • table:insert can then be used to insert records

See: Defining a table

• A function is called for each object found in the dataset:
  • osm2pgsql.process_node - called if the object is a node
  • osm2pgsql.process_way - called if the object is a way
  • osm2pgsql.process_relation - called if the object is a relation

See: Processing callbacks

Script breakdown

Here’s a quick breakdown of the campsites import script:

1. Create tables for the database:

local tables = {}
tables.campsites_all = osm2pgsql.define_table {
  name = "campsites_all",
  -- This will generate a column "osm_id INT8" for the id, and a column
  -- "osm_type CHAR(1)" for the type of object: N(ode), W(way), R(relation)
  ids = { type = 'any', id_column = 'osm_id', type_column = 'osm_type' },
  columns = {
      { column = 'type', type = 'text' },
      { column = 'name', type = 'text' },
      { column = 'tags',  type = 'jsonb' },
      { column = 'geom',  type = 'geometry' },
  }
}

• The tags field is a binary json field that contains additional data captured aboutthe imported feature.
• The geom field will contain the actual geometry of the imported feature. See: geometry, and Spatial Data Model

2. A function to remove uninteresting tags from the data:

function clean_tags(tags)
  tags.odbl = nil
  tags['source:ref'] = nil
  -- tags.created_by = nil
  -- tags.source = nil
  return next(tags) == nil
end

• Returns true if there are no more tags.

3. A function to indicate if a given feature looks like a campsite. This is the core of the filtering:

function looks_like_a_campsite(object)
  return object.tags.tourism == 'camp_site' or object.tags.tourism == 'camp_pitch' or object.tags.tourism == 'caravan_site'
end

• Returns true if the feature’s tags indicate it’s a campsite of some sort.

4. A function to insert a feature into the database:

function process(object, geometry)
  tables.campsites_all:insert({
      type = object.type,
      name = object.tags.name,
      tags = object.tags,
      geom = geometry
  })
end

• The object and geometry are provided separately to the function, and then inserted into the campsites_all table defined earlier.

5. Override osm2pgsql.process_node to process node features:

function osm2pgsql.process_node(object)
  if clean_tags(object.tags) then
      return
  end

  if looks_like_a_campsite(object) then
      process(object, object:as_point())
  end
end

• First cleans out uninteresting object tags, and exits if there are no more tags.
• Checks to see if the object looks like a campsite.
• If acceptable, the object is then inserted into the database with a call to process, and object:as_point() to convert it to a point geometry for PostGIS.

6. Override osm2pgsql.process_way to process way features:

-- Called for every way in the input
function osm2pgsql.process_way(object)
  if clean_tags(object.tags) then
      return
  end
  
  -- object.is_closed is a simple, imperfect, way to check if this is a polygon
  if object.is_closed and looks_like_a_campsite(object) then
      process(object, object:as_polygon())
  end
end

• This is similar to the node processing, except that the way should be closed (eg. a closed polygon).
• object:as_polygon() is used to convert the object to a polygon geometry for PostGIS.

7. Override osm2pgsql.process_relation to process relation features:

function osm2pgsql.process_relation(object)
  if looks_like_a_campsite(object) then
      process(object, object:as_geometrycollection())
  end
end

• Similar to the above, this uses as_geometrycollection to convert the object to a collection geometry for PostGIS.

View a map

pg_tileserv also offers a simple map view, from which you can explore the geometry tables in your PostGIS database.

• Visit: localhost:7800

tables	map

Congratulations!

You have configured, launched and populated a GIS database paired with a tile server. Good luck applying it to your own project!

Security

There are a few security issues to be aware of…

The application is assumed to be running on your personal machine and exposed, at most, to your local home network. In order to use it in any other context, you will need to take some precautionary steps…

The default database password is published in the code repository. This means it is not safe. Do not use this password in production. Modify the values found in: config/word-server.env and do not commit those values to your repository if public.

The tile server runs over HTTP by default. Do not expose this tile server to the internet! Put it behind NGINX or another suitable proxy.

It is also possible to configure pg_tileserv to support SSL certificates.