As part of the 2016 Irish Census the Central Statistics Office captured data on communing to work and school/college (POWSCAR). The raw data is available for research purposes, but the anonymised data is available to all. This data is aggregated to an Electoral Division (ED) level.
There are 3440 EDs in Ireland, 3409 after amalgamating low population ones. These are legally defined administrative areas. Commutes into Northern Ireland are also captured, but only as one destination (Northern Ireland).
Overall the data consists of a CSV file with 291893 rows of commuting data.
Neighborhood boundaries are a fascinating topic. Where do people see their neighborhood extending to, how clear are those boundaries, and how do they shift with time.
This is a topic that has been tackled a number of times around the world, and with different types of locations (like cities). With the general public asked to draw their own neighborhood, or the ones they are familiar with, on a map. But there are challenges.
Sometimes the results are successful, like in Boston. Boston only has around 60 neighborhoods, which are relatively clearly defined based on the mapping results. And Bostonography clearly had a great readership, with over 2300 responses.
Older cities, like Glasgow are much more complex. In 2016 I did a similar survey, and from 367 responses I received 241 unique neighborhoods. You would need a much bigger set of responses to come to any real conclusions. But I still mapped the results for the West End of Glasgow, where most neighborhoods had multiple responses: Glasgow Regions Mapped
The challenges for Dublin neighborhoods is similar to Glasgow, although not as complex. But anything worth doing would require a lot of responses.
Luckily the Dublin InQuirer decided to run a similar survey, polling their subscribers for responses. The number of submissions was good, not quite beating the Boston yet, but getting close. 2200 responses and 133 unique neighborhoods mapped. They also did a few great things. One was making the data available to download. This meant that anyone could map the results, which is great. Additionally they are reaching out to areas with less responses, which is a great effort on their part.
This year I wasn’t really sure I would take part, as I had no plans and nothing prepared. But it is a great challenge. It challenges your creativity, problem solving, and map making skills. It also sets a time limit so you don’t have to worry about being perfect. And making maps is good fun.
There were a few datasets that I had come across that I thought would be good subjects, and I got a lot of mileage out of them.
Dublin neighbourhoods
These included the Dublin Inquirer neighbourhood survey.
I think this is a great initiative, and hopefully they get a real large set of responses. They are at over 2000 already so a great start.
I did something similar for Glasgow (here) but only got 367 responses in total. So the power of having a well read paper behind the initiative is great. They are also reaching out to areas that have not had many responses, which is really great work.
OpenStreetMap
I also wanted to do some maps around OpenStreetMap in Ireland. The community here has had a large push to map all of the buildings in Ireland, which has progressed well.
Supermac’s is an Irish fast food restaurant chain, who have had a few trademark disputes over the years with McDonald’s over the use of Mc and Mac in burger names.
3D:
This turned out a lot better than I expected.
Was pure QGIS. Create grid (5km x 5km), zonal statistics on CORINE (Majority) and DEM (Median). New field for height rounded up to the nearest 40:
to_int(ceil((“_h_median” / 40))) * 40
Set colors. Create centroids with same colors. These become the Lego nubs.
Rendered in QGIS2threejs plugin. The grid is extruded, with a height of height * 50. The centroids are cylinder rendered height * 50 + 30 * 50, so they come a bit higher, radius of 1800.
Might be better to not use landuse, but elevation for the colours.
No Computer:
Nice to get away from the computer. Definitely promotes some creativity. But I just took it as an opportunity to walk on the beach.
Land:
My favourite of my maps.
Land use vineyard across Europe from CORINE 2020, with the major regions labelled.
GHSL:
Seasonal population of the Balearic Islands.
Data clipping in QGIS/GDAL, rendered in Aerialod, with labels with GIMP afterwards.
This was my second favourite of my maps. I think the topic is interesting and the execution is pretty good. Was however quite manual and probably needed more exaggeration to see the differences.
Historical:
A time lapse of 1,831,044 buildings in Ireland being added to OpenStreetMap.
Postcodes were then created based on the ONS Postcode Directory, filtering for postcodes that were live in 2011 (which is the latest census data). The postcode centroids were turned into polygons using voronoi polygons.
Then we simply select all of the buildings in a postcode from Ordnance Survey, Open Map product, filtering out most schools and hospitals. Then we put a random point in a random building for each person in that postcode.
I would have loved to include Northern Ireland, but the Ordnance Survey of Northern Ireland do not have an equivalent open building outline dataset, like Open Map from the Ordnance Survey.
I ran into a process where I wanted to create copies of rasters in PostgreSQL. While seemingly a simple process this took me a bit of work to figure out.
For my workflow I had three rasters, which all have the same size, and I want to load them into the same PostGIS table with three raster geometry columns. I don’t think this will work for different sized rasters since the rid’s will not match.
Three rasters: raster1
raster2
raster3
Which I want to copy into: merged_raster
First to create the merged raster table:
CREATE TABLE merged_raster
(
rid serial NOT NULL,
raster1 raster,
raster2 raster,
raster3 raster
);
Then to add the rid’s. These are the id’s of the tiles that the raster was split into when loading. If your tile size is large enough then you may only have one.
INSERT INTO merged_raster(rid)
(SELECT rid FROM raster1);
Then copying the actual data is straighforward (this assumes the raster column in the raster1 datasets is called rast):
UPDATE merged_raster m
SET raster1 = r.rast
FROM raster1 r
WHERE r.rid = m.rid;
UPDATE merged_raster m
SET raster2 = r.rast
FROM raster2 r
WHERE r.rid = m.rid;
UPDATE merged_raster m
SET raster3 = r.rast
FROM raster3 r
WHERE r.rid = m.rid;
Now I still have an issue that QGIS will not load these layers. It will always load the initial raster column no matter what is chosen.
I was coming across some errors when installing Anaconda, Miniconda specifically.
https://conda.io/miniconda.html
I think part of the reason is I have quite a few installs of Python due to OSGeo4W.
My error:
Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00002554 (most recent call first):
The solution:
Update the activate.bat file that is called when launching from the start menu. For me located in:
C:\ProgramData\Anaconda\Scripts
We need to add in the following to clear out and reset the python environment before launching anaconda:
@REM Test first character and last character of %1 to see if first character is a "
@REM but the last character isn't.
@REM This was a bug as described in https://github.com/ContinuumIO/menuinst/issues/60
@REM When Anaconda Prompt has the form
@REM %windir%\system32\cmd.exe "/K" "C:\Users\builder\Miniconda3\Scripts\activate.bat" "C:\Users\builder\Miniconda3"
@REM Rather than the correct
@REM %windir%\system32\cmd.exe /K ""C:\Users\builder\Miniconda3\Scripts\activate.bat" "C:\Users\builder\Miniconda3""
@REM this solution taken from https://stackoverflow.com/a/31359867
@set "_args1=%1"
@set _args1_first=%_args1:~0,1%
@set _args1_last=%_args1:~-1%
@set _args1_first=%_args1_first:"=+%
@set _args1_last=%_args1_last:"=+%
@set _args1=
@if "%_args1_first%"=="+" if NOT "%_args1_last%"=="+" (
@CALL "%~dp0..\Library\bin\conda.bat" activate
@GOTO :End
)
@CALL "%~dp0..\Library\bin\conda.bat" activate %*
:End
@set _args1_first=
@set _args1_last=
To:
@REM Test first character and last character of %1 to see if first character is a "
@REM but the last character isn't.
@REM This was a bug as described in https://github.com/ContinuumIO/menuinst/issues/60
@REM When Anaconda Prompt has the form
@REM %windir%\system32\cmd.exe "/K" "C:\Users\builder\Miniconda3\Scripts\activate.bat" "C:\Users\builder\Miniconda3"
@REM Rather than the correct
@REM %windir%\system32\cmd.exe /K ""C:\Users\builder\Miniconda3\Scripts\activate.bat" "C:\Users\builder\Miniconda3""
@REM this solution taken from https://stackoverflow.com/a/31359867
@set "_args1=%1"
@set _args1_first=%_args1:~0,1%
@set _args1_last=%_args1:~-1%
@set _args1_first=%_args1_first:"=+%
@set _args1_last=%_args1_last:"=+%
@set _args1=
@SET PYTHONPATH=
@SET PYTHONHOME=C:\ProgramData\Anaconda
@PATH C:\ProgramData\Anaconda;C:\ProgramData\Anaconda\Scripts;%PATH%
@if "%_args1_first%"=="+" if NOT "%_args1_last%"=="+" (
@CALL "%~dp0..\Library\bin\conda.bat" activate
@GOTO :End
)
@CALL "%~dp0..\Library\bin\conda.bat" activate %*
:End
@set _args1_first=
@set _args1_last=
Updating the paths as required.
This just clears out the python and windows environmental variables before launching, similar to what OSGeo4W does.
Save as a text file ending in .xml like qgis_scales.xml
These are the scales OpenStreetMap tiles are rendered in for 96 dpi, so the map will look sharp on most monitors. These are the scales for the zoom levels.
The xml file can then be loaded into the project from:
