Scotland Gender Split

Based on 2011 Census data. We can see a clear majority of the population is Female.

The raw numbers are:
Population total:
5295403
Male total:
2567444
Female total:
2727959
Male total %:
48.48
Female total %:
51.52
Top 5 Male by %:
Shetland Islands – 50.77
Aberdeenshire – 49.52
Orkney Islands – 49.49
Aberdeen City – 49.42
Na h-Eileanan an Iar (Western Isles) – 49.37
Top 5 Female by %
West Dunbartonshire – 52.40
North Ayrshire – 52.37
South Ayrshire – 52.36
East Renfrewshire – 52.34
Inverclyde – 52.14

And the split by local authority:

Scotland Gender Split

X Percent of the Population of Scotland Lives Within Y Miles of Glasgow

I have often heard that X percent of the population Scotland live within Y miles of Glasgow. With the X and the Y varying between claimant.

This is a pretty easy question to answer, using the 2011 Scottish Census population results and the Census Output Area Population Weighted Centroids. Then we get the extents of Glasgow City Council from OS Boundary Line.

The results are:

Pop. Count: %
Scotland 5295403 100
Glasgow 593245 11.2
25 km 2002431 37.8
50 km 2839583 53.6
50 miles 3776701 71.3
100 km 4201860 79.3
100 miles 4483330 84.7

Pretty interesting results, especially the within 50 miles query.

To see how these boundaries look on a map:

Population buffers around Glasgow

A few caveats:
We are using the population weighted centroids, which will produce some minor inaccuracies, but is a very good generalisation.
Also we are using euclidean buffers on the British National Grid plain, so these are not geodesic buffers. The difference will likely be small at these distances.

UK Postcode Polygon Accuracy Comparison Part 2

So we have seen from the previous comparing the raw polygon accuracy between Voronoi generated polygons and NRS generated postcode polygons: Results.

The physical results are interesting, and a visual examination can provide a useful overall comparison, but how does this actually impact me?

I have a CAG from GCC and I just want to attach a postcode to it. How different will my results be between a true postcode boundary dataset from the NRS, and a generated Voronoi dataset from the OS?

I’m glad you are still with me, it might be useful to explain how postcodes actually work in this context:

Lets take a postcode of G31 2XT how does it break down?
Area: G
District: G31
Sector: G31 2
Unit: G31 2XT

So then we can compare how an actual address dataset, like the Glasgow CAG, spatially joined to two postcode datasets compare:

Assuming the NRS dataset is correct (a good assumption) how accurate is a postcode based on an OS Code-Point Open generated Voronoi polygon based on Glasgow City Council residentially classified properties as of 16/11/2014:

Total number of properties:
245096     100%
Correct Area:
245096     100%
Correct District
243650     99.4%
Correct Sector
240956     98.3%
Correct Unit
174344     71.1%

We can see that up to a sector level a Voronoi polygon can produce an extremely accurate results. A visual comparison of how this plays out in Glasgow can be seen here, with the legend best read from the bottom:

UK Postcode Comparison

GIS DAY – Post your Workstation

In honor of GIS day I have posted my current home GIS workstation:

CPU: Intel Xeon E3-1230 V3 3.3GHz Quad-Core Processor
CPU Cooler: Cooler Master Hyper 212 EVO 82.9 CFM Sleeve Bearing CPU Cooler
Motherboard: Gigabyte GA-H97M-D3H Micro ATX LGA1150 Motherboard
Memory: Crucial Ballistix Sport 8GB (1 x 8GB) DDR3-1600 Memory
Memory: Crucial Ballistix Sport 8GB (1 x 8GB) DDR3-1600 Memory
Storage: Crucial MX100 512GB 2.5″ Solid State Drive
Case: Silverstone TJ08B-E MicroATX Mini Tower Case
Power Supply: Corsair CSM 550W 80+ Gold Certified Semi-Modular ATX Power Supply
Monitor: Asus VX279Q 60Hz 27.0″ Monitor

My GIS Battlestation

I love the SSD for speed and the CPU is more that enough for my use.

And for a GIS server I use:
HP 704941-421 ProLiant Micro Server
Running Xubuntu. 3x3tb in a Raid 5 array, with 8gb extra memory.

UK Postcode Polygon Accuracy Comparison

One of the main ways of generating postcode polygons is to use OS Code-Point Open and from them generate Voronoi polygons.

This visualization compares the Code-Point Voronoi polygons to postcodes from NRS postcode extract, Which is widely considered the best postcode dataset for Scotland. Scotland is used because we have a CAG (NLPG in the south) extract for Glasgow available for a property comparison of accuracy.

The black areas are where the two datasets agree and the coloured areas are where they do not. For this comparison we can consider NRS to be correct.

Postcode Comparison

Centroid Within Selection in QGIS

While we have some options for spatial selection in QGIS through the Spatial Query plugin. One option that is glaringly missing is centroid within. This is extremely useful for easily selecting polygons that mainly fall within other polygons. This tutorial will run through how to do a polygon centroid within another polygon selection in QGIS.

Our initial setup is a postcode dataset, where we want to extract all of the ones that are mainly within Glasgow City Council. The boundaries are not the same but are roughly the same. However an intersect query would bring ones that simply touched the area, and a within query would exclude the ones that fall just outside. A centroid within should work great.

Selection

In this image the red lines are our postcodes, and the yellow area is the highlighted Glasgow City polygon.

We are going to cheat slightly by using SpatiaLite, which is a stand alone, single file, spatial database. It is however very tightly integrated into QGIS and we do not have to leave the program so I feel this counts as a QGIS solution.

First using one of the browser panels create a new database:

Screenshot[34]

Transfer your data into the database. This can be done by dragging and dropping a .shp file into the newly created database using two browser panels.

I created a subset of the postcode dataset, with a simple polygon selection of roughly the Glasgow area (postcode_glasgow_nrs_rough). My other dataset is the UK unitary authorities dataset (district_borough_unitary_region).

Then open up the DB Manager. Database>DB Manager>DB Manager.

Once in the database we can do the query using simple SQL:

We also have a WHERE statement so only the ones that within Glasgow are selected. “%lasgow%” used to avoid capitalization mismatches.

Screenshot[35]

We can also directly add this query in as a layer in QGIS using the “Load as new layer” feature. An excellent feature, and only requires you to select the primary key and geometry column. This allows us to visually check our results.

The query has worked as intended, but we have some strangely shaped polygons so the results are not what I had hoped.

Screenshot[37]

We can see that one of the postcode polygons is missing from the selection because its centroid actually falls outside of itself.

Not to worry we have a better option than centroid for this query, which is ST_PointOnSurface. Details can be found on the Boundless PostGIS pages.

So lets try this.

Screenshot[36]

Adding it in we see the results as expected.

Screenshot[38]

So great we now have our data selected, but how do I get it out of SpatiaLite? We could wait for the “Load as new layer” to load in all the features, then save it as a shapefile, but for my query, while great for quick look, the “Load as new layer” was running quite slow and thus not an option.

So instead, we can simply create a new table in the database from our selection.

Note the ; at the end. This creates a new table pretty quickly. And to get it to appear as a spatial table we simply register its geometry in the geometry column:

With the options being: Table name, Geometry column, Geometry (type 6 for polygon), dimensions, SRID, Spatial index boolean.

The table the appears in our browser.

Screenshot[41]

And our final result.

part2

I am loving the database integration in QGIS. It makes some workflows much easier and adds a wealth of new opportunities. Also the “Load as new layer” views are amazing, lots of possibilities.

Open UK Postcode Polygons

The Ordnance Survey releases Code-Point Open, which contains the centroid coordinates for each postcode in the UK. One way to generate open postcode polygons is to generate a Voronoi diagram from those points.

The results initially look good, but how accurate are these generated polygons compared with actual postcode polygons.

Luckily the National Records of Scotland (NRS) also maintain a postcode dataset, which is released on their website for free. So we can do an easy comparison of the two postcode datasets, which should be an indication of how accurate Voronoi postcode areas would be across the UK.

I have decided to use Glasgow for the comparison because we also have the Corporate Address Gazetteer, which will allow us to compare not just the actual polygons, but actual properties. It does not really matter if the postcode polygon is incorrect, if all of the properties within that postcode would still be correct.

Fist we have a simple side by side look at the two datasets we will compare:

Glasgow Postcodes

And a closeup overlay:

Zoomed in

The Voronoi one has been created from Code-Point Open points that fell within the Glasgow City Council Unitary Authority and the output Voronoi was clipped to the extent of the Unitary Authority.

The NRS created postcodes were simply selected from the ones where their “Point on Surface” fell within the Unitary Authority. The process will detailed in a later post.

Polygon Outlines in QGIS

I had an effect of an outline around the grid in the Glasgow Commercial Property Density heatmap. I really like the effect, but to create it I had to save the grid itself as a new file, dissolve it and then symbolise it. There should be a way to do it simply through symbology. When working on some categorised symbology I though symbol levels would be the way to do it, and QGIS 2.6 actually made it possible.

I am not a huge fan of how QGIS deals with polygon boundaries, shared boundaries usually become thicker than non-shared boundaries. But we can still achieve some really good outputs.

We will start out with a simple categorised symbology.

Screenshot[23]

I would like to remove the borders, but that results in very small gaps. We can tell by turning the project background black.

Screenshot[24]

So I will set a narrow white border, which is very unintrusive.

Screenshot[25]

Now for the QGIS 2.6 touch. We can now turn a graduated symbology into a rule based one, simply set up a graduated symbology and then select rule based and your rules will be automatically generated.

Screenshot[26]
Screenshot[27]

For the border we create a catch-all rule that has our desired border.

Screenshot[28]

Resulting in.

Screenshot[29]

But then setting up our symbol levels.

Screenshot[30]

We get the desired result.

Screenshot[31]

Final output.

QGIS Polygon Borders

Desired result with one dataset, excellent.