Categories
All Animations Maps OSGEO PostGIS QGIS Tutorials

My Strava Activities from the Same Place at the Same Time Animated

All of my activities from Strava if they started from the same place at the same time.

Strava Activites Animated

Longest activity time wise is an 11 hour walk.

Distance wise is a 200km helicopter ride.

Includes a couple of 100km cycles. But mainly walks on the beach, 5 km runs, and 10km cycles.

You can see the patterns from a number of locations that I have lived in.

How to

  1. Download your activities from Strava.
  2. Parse them into a PostgreSQL/PostGIS database with Python: https://github.com/HeikkiVesanto/StravaGarminParser/blob/master/parser_points_to_db.py
  3. Process the points to time series in the database: https://gist.github.com/HeikkiVesanto/3fbd55cda45394d069773a34ea244e4b
  4. Create animation in QGIS using the atlas generator.
Categories
All Ireland Maps OSGEO QGIS

Commuting in Ireland Mapped

As part of the 2016 Irish Census the Central Statistics Office captured data on communing to work and school/college (POWSCAR). The raw data is available for research purposes, but the anonymised data is available to all. This data is aggregated to an Electoral Division (ED) level.

There are 3440 EDs in Ireland, 3409 after amalgamating low population ones. These are legally defined administrative areas. Commutes into Northern Ireland are also captured, but only as one destination (Northern Ireland).

Overall the data consists of a CSV file with 291893 rows of commuting data.

Maps inspired by Dónal Casey.

Ireland

Mapping Irish commuting data at a country level allows us to see the patterns of commuting.

Purple

The map styles can easily be achieved in QGIS. They utilise blending modes and variable levels of symbology based on the commuter counts.

A great guide can be found on this from the Ordnance Survey. Carto tips: Using blend modes and opacity levels

I also talk about this at my FOSS4GUK 2019 talk: YouTube – IE
FOSS4GUK LIVE – Main Room / Biosphere Green – DAY 2

The purple map above use color: #6450d7 – Addition blend – 100% opacity – Line thickness: if (“count”>1000, 1000, “count”)/1000

Gold:

Gold

Color: #e5b636 – Addition blend – 100% opacity

This map was featured in the 2020 GeoHipster calendar.

Black and White:

9 cities

Nine largest cities:

9 cities

Individual Cities

I have also created individual maps for all settlements in Ireland. This should give a better local picture of the commuter flows.

Settlements were linked to EDs based on a few characteristics:

  • Settlement intersects ED AND either
  • The original point on surface is within 100 meters of a settlement OR
  • 75% of the settlement is in the ED OR
  • 90% of the ED is in the settlement

List of all settlements.

Examples:

Letterkenny:

Letterkenny

Galway:

Galway

Castleisland:

Castleisland

Tramore:

Tramore
Categories
All Featured Ireland Maps OSGEO PostGIS QGIS

Dublin Neighborhoods Mapped

Neighborhood boundaries are a fascinating topic. Where do people see their neighborhood extending to, how clear are those boundaries, and how do they shift with time.

This is a topic that has been tackled a number of times around the world, and with different types of locations (like cities). With the general public asked to draw their own neighborhood, or the ones they are familiar with, on a map. But there are challenges.

Sometimes the results are successful, like in Boston. Boston only has around 60 neighborhoods, which are relatively clearly defined based on the mapping results. And Bostonography clearly had a great readership, with over 2300 responses.

Older cities, like Glasgow are much more complex. In 2016 I did a similar survey, and from 367 responses I received 241 unique neighborhoods. You would need a much bigger set of responses to come to any real conclusions. But I still mapped the results for the West End of Glasgow, where most neighborhoods had multiple responses: Glasgow Regions Mapped

The challenges for Dublin neighborhoods is similar to Glasgow, although not as complex. But anything worth doing would require a lot of responses.

Luckily the Dublin InQuirer decided to run a similar survey, polling their subscribers for responses. The number of submissions was good, not quite beating the Boston yet, but getting close. 2200 responses and 133 unique neighborhoods mapped. They also did a few great things. One was making the data available to download. This meant that anyone could map the results, which is great. Additionally they are reaching out to areas with less responses, which is a great effort on their part.

Dublin InQuirer – Will You Draw Your Dublin Neighbourhood for Us?

Map your neighborhood.

Results:

I have mapped all of the neighborhoods as an interactive map, and individually if they had 10 or more responses.

Interactive:

Full page.

Individual Dublin Hoods:

The following have been mapped as they had over 10 responses:

Artane,
Baldoyle,
Ballinteer,
Ballsbridge,
Ballybough,
Ballyfermot,
Beaumont,
Blackrock,
Broadstone,
Cabra,
Castleknock,
Churchtown,
Clondalkin,
Clontarf,
Crumlin,
Dolphins Barn,
Donnybrook,
Donnycarney,
Drimnagh,
Drumcondra,
Dun Laoghaire,
East Wall,
Fairview,
Finglas,
Firhouse,
Glasnevin,
Goatstown,
Harold’s Cross,
Inchicore,
Islandbridge,
Kilmainham,
Kimmage,
Knocklyon,
Lower Crumlin,
Lucan,
Malahide,
Marino,
Navanroad,
Phibsborough,
Portmarnock,
Portobello,
Raheny,
Ranelagh,
Rathfarnham,
Rathgar,
Rathmines,
Rialto,
Sandymount,
Santry,
Smithfield,
Stoneybatter,
Sutton,
Tallaght,
Terenure,
The Liberties,
The Tenters,
Whitehall

Creation:

These maps were created with Python for downloading the data, PostgreSQL/PostGIS for the data processing, and QGIS for the rendering.

There is not a full tutorial, but the processing code is available in here:

Downloading the data.

SQL processing.

Categories
30 Day Map Challenge All Featured Ireland Maps OpenStreetMap OSGEO QGIS Scotland

30 Day Map Challenge 2021

This November I once again took part in the 30 Day Map Challenge started by Topi Tjukanov.

Categories

I had done it the first year (30 Day Map Challenge 2019), and had made a few maps for 2020 as well.

This year I wasn’t really sure I would take part, as I had no plans and nothing prepared. But it is a great challenge. It challenges your creativity, problem solving, and map making skills. It also sets a time limit so you don’t have to worry about being perfect. And making maps is good fun.

There were a few datasets that I had come across that I thought would be good subjects, and I got a lot of mileage out of them.

Dublin neighbourhoods

These included the Dublin Inquirer neighbourhood survey.

Will You Draw Your Dublin Neighbourhood for Us?

neighbourhoods.dublininquirer.com

I think this is a great initiative, and hopefully they get a real large set of responses. They are at over 2000 already so a great start.

I did something similar for Glasgow (here) but only got 367 responses in total. So the power of having a well read paper behind the initiative is great. They are also reaching out to areas that have not had many responses, which is really great work.

OpenStreetMap

I also wanted to do some maps around OpenStreetMap in Ireland. The community here has had a large push to map all of the buildings in Ireland, which has progressed well.

But it is very much a work in progress.

OSM Ireland Buildings poster from State Of The Map 2021

Selected Maps

Hexagons:

Hex

First of the Dublin boundaries series.

Final interactive version: Here

OpenStreetMap:

OpenStreetMap

For OSM day I wanted to try and make the data a bit easier to use for #QGIS novices.

I created styles that can be applied to the GEOFABRIK Shapefile extracts, from here.

The styles are available: https://github.com/HeikkiVesanto/QGIS_OSM_Styles

Red:

Red

Mac vs Mc.

Supermac’s is an Irish fast food restaurant chain, who have had a few trademark disputes over the years with McDonald’s over the use of Mc and Mac in burger names.

3D:

3D

This turned out a lot better than I expected.

Was pure QGIS. Create grid (5km x 5km), zonal statistics on CORINE (Majority) and DEM (Median). New field for height rounded up to the nearest 40:

to_int(ceil((“_h_median” / 40))) * 40

Set colors. Create centroids with same colors. These become the Lego nubs.

Rendered in QGIS2threejs plugin. The grid is extruded, with a height of height * 50. The centroids are cylinder rendered height * 50 + 30 * 50, so they come a bit higher, radius of 1800.

The “rayshader” export makes it look realistic.

Interactive: https://maps.gisforthought.com/LegoIreland/

Might be better to not use landuse, but elevation for the colours.

No Computer:

No Computer

Nice to get away from the computer. Definitely promotes some creativity. But I just took it as an opportunity to walk on the beach.

Land:

Land

My favourite of my maps.

Land use vineyard across Europe from CORINE 2020, with the major regions labelled.

GHSL:

GHSL

Seasonal population of the Balearic Islands.

Data clipping in QGIS/GDAL, rendered in Aerialod, with labels with GIMP afterwards.

This was my second favourite of my maps. I think the topic is interesting and the execution is pretty good. Was however quite manual and probably needed more exaggeration to see the differences.

Historical:

Historical

A time lapse of 1,831,044 buildings in Ireland being added to OpenStreetMap.

If you want to get involved see: OpenStreetMap Ireland Buildings

Globe:

Globe

Simple spinning globe in QGIS, but I was happy that I was able to automate the export: Gist

All maps:

See here

Categories
All Featured Ireland Maps OSGEO PostGIS QGIS Scotland

Every Person in Great Britain Mapped

A follow up to my previous post: Every Person in Scotland on the Map. Winner of the 2016 OS OpenData Award for Excellence in the use of OpenData from the British Cartographic Society.

Full size interactive map.

The mapping process is pretty straightforward, and not accurate. I don’t know where you live. But I can make an educated guess.

I simply amalgamate the two sets of census data from the NRS (National Records of Scotland) for Scotland (2011 census) and the ONS (Office of National Statistics) for England and Wales (2010 census).

Postcodes were then created based on the ONS Postcode Directory, filtering for postcodes that were live in 2011 (which is the latest census data). The postcode centroids were turned into polygons using voronoi polygons.

Then we simply select all of the buildings in a postcode from Ordnance SurveyOpen Map product, filtering out most schools and hospitals. Then we put a random point in a random building for each person in that postcode.

I would have loved to include Northern Ireland, but the Ordnance Survey of Northern Ireland do not have an equivalent open building outline dataset, like Open Map from the Ordnance Survey.

Rendered with: QGIS tile writer python script. Processing done 100% in PostGIS.

Categories
All Maps OSGEO QGIS Tutorials

Mapping Strava Data

In 2018 we started a running club at work.

I created a quick script to parse the data on Strava to a ShapeFile, which can be easily animated with QGIS.

The script only works with Garmin files, GPX, TCX, and FIT.

Source: https://github.com/HeikkiVesanto/Strava-Garmin-Parser

Example:

Categories
All Animations Ireland Maps OpenStreetMap OSGEO

OpenStreetMap Ireland 2018 Visualized

2018 was a very productive year for OpenStreetMap in Ireland.

Around 70k nodes added, 60k ways added, and 160k polygons added.

Finally a company was formed to facilitate becoming a fully fledged local chapter.

What does that look like day to day:

Categories
All OSGEO PostGIS QGIS

Copying Rasters in PostGIS

I ran into a process where I wanted to create copies of rasters in PostgreSQL. While seemingly a simple process this took me a bit of work to figure out.

For my workflow I had three rasters, which all have the same size, and I want to load them into the same PostGIS table with three raster geometry columns. I don’t think this will work for different sized rasters since the rid’s will not match.

Three rasters:
raster1
raster2
raster3

Which I want to copy into:
merged_raster

First to create the merged raster table:

CREATE TABLE merged_raster

(

  rid serial NOT NULL,

  raster1 raster,

  raster2 raster,

  raster3 raster

);

Then to add the rid’s. These are the id’s of the tiles that the raster was split into when loading. If your tile size is large enough then you may only have one.

INSERT INTO merged_raster(rid)

(SELECT rid FROM raster1);

Then copying the actual data is straighforward (this assumes the raster column in the raster1 datasets is called rast):

UPDATE merged_raster m

SET raster1 = r.rast

FROM raster1 r

WHERE r.rid = m.rid;



UPDATE merged_raster m

SET raster2 = r.rast

FROM raster2 r

WHERE r.rid = m.rid;



UPDATE merged_raster m

SET raster3 = r.rast

FROM raster3 r

WHERE r.rid = m.rid;

Now I still have an issue that QGIS will not load these layers. It will always load the initial raster column no matter what is chosen.

Categories
All OSGEO

Anaconda install with OSGeo4W

I was coming across some errors when installing Anaconda, Miniconda specifically.

https://conda.io/miniconda.html

I think part of the reason is I have quite a few installs of Python due to OSGeo4W.

My error:

Fatal Python error: Py_Initialize: unable to load the file system codec

ModuleNotFoundError: No module named 'encodings'



Current thread 0x00002554 (most recent call first):

The solution:

Update the activate.bat file that is called when launching from the start menu. For me located in:

C:\ProgramData\Anaconda\Scripts

We need to add in the following to clear out and reset the python environment before launching anaconda:

@SET PYTHONPATH=

@SET PYTHONHOME=C:\ProgramData\Anaconda

@PATH C:\ProgramData\Anaconda;C:\ProgramData\Anaconda\Scripts;%PATH%

So, editing the file from:

@REM Test first character and last character of %1 to see if first character is a "

@REM   but the last character isn't.

@REM This was a bug as described in https://github.com/ContinuumIO/menuinst/issues/60

@REM When Anaconda Prompt has the form

@REM   %windir%\system32\cmd.exe "/K" "C:\Users\builder\Miniconda3\Scripts\activate.bat" "C:\Users\builder\Miniconda3"

@REM Rather than the correct

@REM    %windir%\system32\cmd.exe /K ""C:\Users\builder\Miniconda3\Scripts\activate.bat" "C:\Users\builder\Miniconda3""

@REM this solution taken from https://stackoverflow.com/a/31359867

@set "_args1=%1"

@set _args1_first=%_args1:~0,1%

@set _args1_last=%_args1:~-1%

@set _args1_first=%_args1_first:"=+%

@set _args1_last=%_args1_last:"=+%

@set _args1=





@if "%_args1_first%"=="+" if NOT "%_args1_last%"=="+" (

    @CALL "%~dp0..\Library\bin\conda.bat" activate

    @GOTO :End

)



@CALL "%~dp0..\Library\bin\conda.bat" activate %*



:End

@set _args1_first=

@set _args1_last=

To:

@REM Test first character and last character of %1 to see if first character is a "

@REM   but the last character isn't.

@REM This was a bug as described in https://github.com/ContinuumIO/menuinst/issues/60

@REM When Anaconda Prompt has the form

@REM   %windir%\system32\cmd.exe "/K" "C:\Users\builder\Miniconda3\Scripts\activate.bat" "C:\Users\builder\Miniconda3"

@REM Rather than the correct

@REM    %windir%\system32\cmd.exe /K ""C:\Users\builder\Miniconda3\Scripts\activate.bat" "C:\Users\builder\Miniconda3""

@REM this solution taken from https://stackoverflow.com/a/31359867

@set "_args1=%1"

@set _args1_first=%_args1:~0,1%

@set _args1_last=%_args1:~-1%

@set _args1_first=%_args1_first:"=+%

@set _args1_last=%_args1_last:"=+%

@set _args1=



@SET PYTHONPATH=

@SET PYTHONHOME=C:\ProgramData\Anaconda

@PATH C:\ProgramData\Anaconda;C:\ProgramData\Anaconda\Scripts;%PATH%



@if "%_args1_first%"=="+" if NOT "%_args1_last%"=="+" (

    @CALL "%~dp0..\Library\bin\conda.bat" activate

    @GOTO :End

)



@CALL "%~dp0..\Library\bin\conda.bat" activate %*



:End

@set _args1_first=

@set _args1_last=

Updating the paths as required.

This just clears out the python and windows environmental variables before launching, similar to what OSGeo4W does.

Categories
All OpenStreetMap OSGEO QGIS

QGIS OpenStreetMap Scales

Save as a text file ending in .xml like qgis_scales.xml

These are the scales OpenStreetMap tiles are rendered in for 96 dpi, so the map will look sharp on most monitors. These are the scales for the zoom levels.

The xml file can then be loaded into the project from:

Project> Project Properties…> General> Project scales

<qgsScales version="1.0">
    <scale value="1:554678932"/>
    <scale value="1:277339466"/>
    <scale value="1:138669733"/>
    <scale value="1:69334866"/>
    <scale value="1:34667433"/>
    <scale value="1:17333716"/>
    <scale value="1:8666858"/>
    <scale value="1:4333429"/>
    <scale value="1:2166714"/>
    <scale value="1:1083357"/>
    <scale value="1:541678"/>
    <scale value="1:270839"/>
    <scale value="1:135419"/>
    <scale value="1:67709"/>
    <scale value="1:33854"/>
    <scale value="1:16927"/>
    <scale value="1:8463"/>
    <scale value="1:4231"/>
    <scale value="1:2115"/>
</qgsScales>

Example:

1,000,000 (QGIS default):

1,083,357 (OSM wiki):

1,155,584 (From: 3liz):

Scales from:
OSM wiki

Zoom scales.