Creating OpenStreetMap History Visualisations

I created a couple of OSM visualisations for my talk at the OSGeo Ireland conference.

See: History of OpenStreetMap in Ireland

These are pretty easy to make, but take a fair bit of time. I did mine for Ireland, but should work with any part of the world.

Required software:

This is the trickiest part, installing osmium-tools: here.

Data:

An OSM full history export. The best source for these is GEOFABRIK.

For Ireland:

http://download.geofabrik.de/europe/ireland-and-northern-ireland.html

Due to GDPR, you will have to log in with an OSM id to download the full history extracts. User ID’s are personal data.

Process:

The workflow is pretty simple. Osmium-tools provides pretty easy API access to the history files, where you can provide a data, and it will extract what OSM was like at that date. We simply need to loop through the desired dates we want to extract, and pipe the results into a workflow that loads the data into PostgreSQL. The final step is simply rendering in QGIS using the time manager plugin.

Python Script:

Github GIST:

https://gist.github.com/HeikkiVesanto/f01ea54cca499a6a144d18cf8909c940

The tables in the database will be:

  • lines
  • multilinestrings
  • multipolygons
  • other_relations
  • points

Each feature will be tagged with the date it is associated with.

Visualisation:

To visualise the data in QGIS we use simply use the excellent time manager plugin, filtering on the load_date field and with a monthly interval.

Result:

History of OpenStreetMap in Ireland

Last week I was invited to give a workshop at the Second Irish OSGeo Conference in Portlaoise. The event was a great success with a attendees from across the OSGeo space, from academics to startups.

I also volunteered to give a talk on OpenStreetMap (OSM) in Ireland. Since we are currently in the process of setting up an official chapter of OSM in Ireland. Check out OpenStreetmaMap.ie if you want to help.

Check out my full talk: here

But some of the visuals are more interesting than the talk as a whole.

To start out, the first edits in Ireland.

The first line:

Located in Banbridge, in Northern Ireland between Newry and Belfast:

First polygon:

Clearly St. Stephens Green, a great park in Dublin:

Then a look at the full history of roads in Ireland on OpenStreetMap:

A large part of the history of OpenStreetMap in Ireland is the townlands project. Townlands in Ireland are small divisions of land, often used in addressing for example. Your address could be: Mr. O’Brien, Blue house, Tawny (the townland), Donegal. The postman knows where you are.

There are over 61,000 townlands in Ireland. Traced manually from 650+ out of copyright OSi maps from Trinity College. Check out Townlands.ie for more info.

The project ran from 2012 to 2017 and the progress is clear when charting the history of boundaries in OpenStreetMap in Ireland:

And since the conference was held in Portlaoise, here is how it looks over time in OSM:

For the script to generate these histories see: Creating OpenStreetMap History Visualisations

QGIS Select Within Plugin 0.4

Version 0.4 brings a couple of new features to the Select Within plugin.

Most importantly it now supports mostly within and percentage within selections.

Also introduced is Pole of Inaccessibility within selections. As implemented in the QGIS core with the Polylabel algorithm: https://github.com/mapbox/polylabel

Point within selection examples. Centroid (red), Point on Surface (green), and Pole of Inaccessibility (blue):

Examples:

Centroid within:

Point on Surface within:

Pole of Inaccessibility within (1.0 tolerance):

50% within:

75% within:

Code:
GitHub Code Base

Issues and bug tracker:
GitHub Issues

QGIS Multi Ring Buffer Plugin Version 1

After about 3 years of existing. I am releasing version 1 of the Multi Ring Buffer Plugin.

QGIS plugin repository.

With version 1 comes a few new features:

  • Ability to choose the layer to buffer after launching the plugin
  • Ability to supply buffer distances as comma separated text string
  • Ability to make non-doughnut buffers
  • Doughnut buffer:

    Non-doughnut buffer (regular):

    Updating A Plugin From QGIS 2 to QGIS 3

    I have two plugins in the QGIS plugin repository, and with the release of QGIS 3 looming it was time to upgrade them for QGIS 3.

    There is a short guide by the QGIS dev team that is a good starting point at:
    https://github.com/qgis/QGIS/wiki/Plugin-migration-to-QGIS-3

    But I had not done any development on these plugins for a while so a more step by step guide was useful, so hopefully, write the guide for the first plugin and follow it step by step for the second.

    I am working on Windows, with OSGeo4W.

    Before we start we will need to insure a couple of extras are installed through the OSGeo4w Installer:
    Desktop:
    qgis-dev
    Libs:
    python-future

    Assuming GitHUB is the repo.
    In git shell:

    There is a conversion script for QGIS plugins provided by the QGIS devs in the main repo.

    UPDATE:
    The qgis2to3 packages can be found on pip now:

    https://github.com/opengisch/qgis2to3/

    We can download just the scripts folder using the following link:
    https://minhaskamal.github.io/DownGit/#/home?url=https://github.com/qgis/QGIS/tree/master/scripts

    Extract that into a location of your choice.

    Then we can run the 2to3 script from the OSGeo4W console (cd to the folder you extracted the script to):

    This will print out changes that need to be made to convert from QGIS2 to QGIS3.

    My first run resulted in many lines of:

    So my plugins line of:

    Was impossible to convert with the tool, since I was not 100% sure what I needed from the QtCore library (I was young when I wrote the plugin). I commented out the line for the plugin in QGIS 2.8, booted up QGIS 2.8 and tried running the plugin.

    So python errors:
    NameError: global name ‘QVariant’ is not defined
    NameError: global name ‘Qt’ is not defined
    Later. I ended up expanding my other import from QtCore to:

    Running the 2to3 script again looked ok, with a number of changes required. These changes can be applied with –w flag:

    For the next step I booted up my favourite IDE PyCharm. I created a bat file that launched PyCharm with the QGIS dev environmental variables. So copying the “python-qgis-dev.bat” file from:

    I changed the final line of:

    To:

    Then from File> Settings> Project:> Project Interpreter> Set to “C:OSGeo4W64appsPython36python.exe”

    It takes a while to update the interpreter.

    I only had 2 errors, both for:
    QgsMapLayerRegistry.instance().addMapLayer(vl)

    There is a list of API breaks between QGIS 2 and QGIS 3 at:
    https://qgis.org/api/api_break.html

    Looks like QgsMapLayerRegistry was moved to QgsProject. So I edit it to:

    Then we can edit the metadata.txt to QGIS 3:
    qgisMinimumVersion=3.0

    And increase the version number.

    Then we need to recompile the icon and ui for Python3 and QT5.

    I was struggling a bit with the environmental variables to get it working, and ended up using a great batch script form StackExchange:
    https://gis.stackexchange.com/questions/260743/how-to-compile-qtdesigner-user-interface-ui-and-resource-qrc-files-with-qg

    So create the .bat file and run it in the folder of you plugin (editing where needed). Note: Your resources_rc may be called resource_rc or something slightly different.

    Move the plugin folder to:
    C:\Users\USERNAME\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\

    Boot up QGIS2.99/3.

    I had a few more issues.

    It seems QGIS 3 deals with the icon slightly differently.

    icon_rc.py is no longer needed, and it seems was not used on my other plugin either.

    So I removed the reference to it in the main python script:
    from . import icon_rc

    I still had some errors.

    AttributeError: module ‘qgis.PyQt.QtGui’ has no attribute ‘QDialog’

    It seems QDialog has moved to PyQt.QtWidgets.

    So in my multi_ring_buffer_dialog.py file I needed to change some lines:

    Add:

    Change:
    QtGui.QDialog
    to:
    QDialog
    In the two instances in that file.

    Working plugin!

    Commit the changes back to the repo. Cd to the directory in git shell.

    Zip the plugin up.
    Upload to https://plugins.qgis.org/plugins/

    Second plugin:
    Same issue with import *
    1 error with QgsMapLayerRegistry
    My resources_rc file was called resource_rc so the batch script needed to be edited to:
    call pyrcc5 resources.qrc -o resource_rc.py
    Same issues with QtGui.QDialog

    Now time for some improvements.