Visiting the Great Polish Map of Scotland

The Great Polish Map of Scotland is located on the grounds of the Barony Castle Hotel. Found near the borders town of Eddleston, just north of Peebles, or a 50 minute drive from Edinburgh. We visited it on our way north from England, which didn’t take use too much out of our way. It is free to visit so a must visit for all map enthusiasts in Scotland or Northern England.

The Great Polish Map of Scotland GIF

Built between 1974 and 1979 to a scale of 1:10000 it is truly impressive to behold (50 m x 40 m on the ground). It is the largest known physical representation of a country and the largest outdoor relief map in the world (signpost at the map). Envisioned by Jan Tomasik, a Polish sergeant. He had been station in Galashiels during WW2, and added the Hotel Black Barony to his holdings in 1968, providing the map a location. The map itself was built in stages over a number of years, with Polish exchange students providing some of the manpower required.

The map is now maintained by Mapa Scotland. A charity who are now working on restoring the map. Parts of it are currently in poor condition, but overall it is an impressive site.

Also the grounds of the hotel in general are really beautiful, with a small glen running through the site, if visiting the Scottish Borders it would make a great place to stay: Barony Castle Hotel.

When arriving at the hotel there is a path just to the left of the castle:
Barony Castle Hotel The path continues on the bridge over the glen:
Bridge to the map. Finally the map, the white stick is the location of Ben Nevis. Looking from the east, so Aberdeen and Aberdeenshire in the forefront:
The Great Polish Map of Scotland View from the west, with the Western Isles in the forefront:
The Great Polish Map of Scotland View from the south with the Scottish Borders and Dumfires and Galloway in the forefront:
The Great Polish Map of Scotland Closeup of the details, the erosion on the edges is evident:
The Great Polish Map of Scotland

YouTube:

Video 1 and Video 2.

Bing:

Bing Maps Bird’s eye view has a good image of the map.

Google:
Location on Google Maps.

For more information please visit:
Mapa Scotland
The Great Polish Map of Scotland on Wikipedia
BBC News

UK Rail Network Visualized by Operator

The United Kingdom’s railway network is split up into regional franchises, these franchises are run by private companies who bid on the work. So for example ScotRail, which runs the trains in Scotland is owned by the FIrst Group, who also run a number of other regional franchises. This can however change, when the contracts are up for bid.

Interestingly there is also a governmental holding company (Directly Operated Railways), which can take over a franchise if necessary, they currently run East Coast Trains.

For this visualization I have grouped Serco-Abellio and Abellio as Serco/Abellio.

EDIT: Like mentioned in the Scotland to the Rest of the UK, lines are drawn between stations on a route. So a direct route from Glasgow to London would be a direct line between the two terminus stations.

UK Railway Network by Operator

Inspired by Scotland’s connections to the rest of the UK.

Data:

Train lines: GTFS Data Exchange

Country outline: Natural Earth

Public Transport Connections Between Scotland and the Rest of the UK

As a follow up to Scotland’s connections to outside the UK, here are the connections between Scotland and the rest of the UK. Missing is the bus network.

The train lines are drawn between stops on the route, so a direct train from London to Glasgow would be a straight line between the two terminus stations.

The real interesting thing is that there is a crosscountry train running from Glasgow to Penzance.

Transport_to_UK

Data sources:

Flights – OpenFlights.org

Trains – GTFS Data Exchange

Ferries - OpenStreetmap

Land – NaturalEarth

Scotland’s Connections to the Wider World

On the verge of the Scottish vote for independence here is a look at all of the scheduled passenger connections between Scotland and outside of the UK. As it is, there are only flights. There are a few ferries between Northern Ireland and Scotland, however NI is still within the UK. No conclusions just an interesting visualisation.

Flights from Scotland to outside the UK

Thanks to:

The World Is A Village - PostGIS: using latitude and longitude to create geometry

Anita Graser - Visualizing Global Connections

Data from:

Natural Earth

OpenFlights

Scotland’s changing outline

In 1654 Joannis Blaeu published volume 5 of his Atlas Novus. The Atlas contained three general maps of Scotland and 46 maps of Scottish counties or regions, making Scotland the best mapped country in the world.

The Blaeu outline was influential on the outline of Scotland for the next decade to come. This is a comparison of subsequent outlines of Scotland before the 1747-1755 Roy Military Survey of Scotland truly surveyed the whole of Scotland.

The maps in this comparison are:

Excluded is the [1687] – Robert Morden – A mapp of Scotland made by R. Gordon because it essentially just follows the Blaeu outline.

In addition a background map was created from OS BoundaryLine (High Water Polyline) and converted to a polygon.

Everything is converted to WGS 84 / World Mercator projection (EPSG:3395).

For more on how the outlines were created:

Georeferencing vector data

For a history of the maps used:

Historic Maps of Scotland from Blaeu to Dorret (1600-1700)

If you are interested in buying a historically amazing map:

Blaeu Atlas Maior of 1665 - Including the atlas of Scotland.

Scotland’s changing coastline:

Scotlands Changing Coastline

Historic Maps of Scotland from Blaeu to Dorret (1600-1700)

Introduction

The Blaeu Atlas of Scotland can be considered the first Atlas of Scotland; published in volume 5 of Joannis Blaeu’s 1654 Atlas Novus, it contained three general maps of Scotland and 46 maps of Scottish counties or regions. The Blaeu atlas turned Scotland into one of the best-mapped countries in the world at the time. The outline of Scotland created by Robert Gordon (the Gordon-Blaeu outline), mainly from William Pont’s sketches, is said to have heavily influenced maps of Scotland for the next decade to come until the 1747-55 William Roy Military Survey of Scotland. The accuracy of the Roy survey makes looking at further changes less interesting. A look at the maps of Scotland prior to the Blaeu atlas could be interesting, but the speculation in the mapping makes a comparison more difficult.

This post is based on my undergraduate dissertation “The influence of the Blaeu Atlas of Scotland on subsequent maps of Scotland”, University of Glasgow, Department Of Geographical And Earth Sciences, 2009. A copy of which should be found in the said department.

The National Library of Scotland (NLS) have done a spledid job with making historic maps available to the general public. Not only are they available online, they are being promoted, and georeferenced.

I love the Blaeu Atlas of Scotland and this post will give a bit of background on the atlas and its influence on the subsequent maps of Scotland:

Overview:

Jeffery C. Stone, in his unpublished Ph.D. thesis on the Blaeu Atlas of Scotland, describes the cartography of Scotland before the 19th century as: “Brief phases of remarkable achievement … interspersed with much longer periods when little progress can be discerned” (Stone, 1972, p.1). The Blaeu Atlas of Scotland is one of those remarkable achievements. The Atlas of Scotland makes up just one part of volume V of Blaeu’s Theatrum Orbis Terrarum sive Atlas Novus, his 11 or 12 volume Atlas Maior covering the whole world (Fleet, 2005). Volume V contains 56 maps and around 200 pages of text, depending on the version. Forty-nine of the maps are of Scotland and the remaining 6 are of Ireland. Of the Scottish maps 46 are regional maps, one is a particularly inaccurate map of Great Britain and Ireland, and two are maps of the whole of Scotland. The maps of Scotland consist of “Scotia Antiqua” (Old Scotland) and “Scotia Regnum” (Kingdom of Scotland). The two maps contain only minor differences in coastline and geographical features, differing primarily only in place names. The Blaeu Atlas of Scotland is a remarkable achievement in the history of Scottish cartography because through it Scotland became one of the best mapped countries of the seventeenth-century world” (Stone, 1991, p. 5). When the sixth volume of his Atlas Novus was released in 1655, which included maps of China, the maps of Scotland formed one eighth of the total maps in his world atlas.

Maps:

[1654] – Joan Blaeu – Atlas of Scotland

For a full history of the composition and editing of the Blaeu Atlas of Scotland see:

  • Stone (1970, 1972, 1981 and 1989)
  • Cash (1901 and 1907)
  • Moir and Skeleton (1968)

For a great introduction see: The history behind the publication of the Blaeu Atlas of Scotland by the NLS

For further reading: Blaeu Atlas of Scotland, 1654

The origins of the forty-six regional maps contained in Blaeu’s Atlas of Scotland published in 1654, may go back as much as seventy years prior to publication (Stone, 1970, p.16). Other Amsterdam cartographers knew something of Pont’s surveys by 1610 when Jodocus Hondius II (d. 1612) engraved a map entitled ‘A new Description of the Shyres Lothian and Linlitquo Be T. Pont’. This one engraving is attributed to Andrew Hart, an Edinburgh bookseller who had only one map of Pont’s engraved in Amsterdam (Koeman, 1970, p. 70). The surveys for the Blaeu maps can be attributed to Timothy Pont, Robert Gordon, James Gordon, and the Earl of Sutherland. With further surveys possibly conducted by others, notably a marine chart of the Tay Estuary, which had been long attributed to Pont was likely done by James Mar (Stone, 1989, p.210). The final engraved maps of the Blaeu atlas however rely heavily on the original work by Pont, which were likely conducted during the summers between 1583 and 1601 (The Early Maps of Scotland, 1973, p.41). Stone’s extensive research on the subject suggests that thirty-three of the forty-seven maps were engraved directly from Pont manuscripts (Stone, 1989 p.13). Nine were drafted by Robert Gordon from Pont sources without significant alteration and a further three by incorporating new material as well as information from Pont manuscripts. One is by James Gordon and the final map of Lorna is unknown, but it is dedicated by Blaeu to Sir James Balfour.

[1654] – Joan BLAEU – Scotia regnum

The outline of Scoland “Scotia Regnum” is attributed to Robert Gordon on the map itself, and the resulting outline of Scotland was coined by Inglis et al. (1934, p. 11) as the Gordon-Blaeu outline. Inglis et al. call the outline “an entirely new outline of Scotland, which for the next hundred years became the standard map” (1934 p. 11). The Maps of Scotland (1973) is more critical of the Blaeu-Gordon outline: “Gordon’s outline of Scotland is markedly less accurate in many respects [than Mercator's map of 1595], although the interior is much improved and more detailed and the number of place names vastly increased” (The maps of Scotland, 1973, pp. 51-52). Specifically the Gordon-Blaeu outline has more noticeable errors in the shape of Lewis, with its flat top, the distortion of the north coast, and the bend in the Great Glen. Cartographers copied these errors for the next ninety years. Withers (2005, p.305) attributes the Gordon-Blaeu outline for shaping the mapped outlines of the nation for nearly another one hundred and forty years. This is despite considering the outline to be less accurate in several respects than earlier maps, which he suggests is a result of the misinterpretation by Gordon of Pont’s survey work.

1665 – Nicolas Sanson – L’Escosse Royaume

According to Inglis et al. (1934, p. 28) Sanson used the Blaeu outline but was careless in execution leading to a less accurate map. The Early Maps of Scotland (1973, p.55) on the other hand puts forward that Sanson used the county and regional maps to create a new and better map. The influence of the regional maps is seen particularly in the coastline of Galloway and of the Moray Firth. Sanson had to rely on the Gordon-Blaeu maps for his north coast and the misshapen isle of Lewis, due to a lack of more accurate sources. Withers (2005, p. 303) is in accord with Inglis et al., stating that Sanson clearly drew from the Blaeu-Gordon outline. The National Library of Scotland (NLS, 2009 a) suggests that Sanson used both the Scotia Regnum (1654) and regional maps by Pont. According to the National Library of Scotland, Sanson’s map went on to have a major influence on the maps of Frederick de Wit, Jean-Baptiste Nolin, Nicolaus Visscher, and Carel Allard in the 1680s and 1690s.

[1687] – Robert Morden- A mapp of Scotland made by R. Gordon

Withers (2005, p. 303) points out that Morden himself states that he has “corrected and improved” the Gordon-Blaeu outline. The only differences according to Withers are: “A grid-square has been overlaid in addition to the alphabetical tables, colour has been added to the sheriffdoms and counties, Rothesay is shown on ‘Boot’ [Bute] on Morden and not on Blaeu” (2005, p. 303). Inglis et al. (1934, p. 29) and The Early Maps of Scotland (1973, p. 82) are in agreement, calling Morden a careful copy, even down to the place names, their position on the map and the style of lettering. The National Library of Scotland (NLS, 2009 a) does suggest that the Mercator-derived map of John Speed may have also influenced Morden.

1714 – Hermann Moll – The north part of Great Britain called Scotland

Herman Moll’s 1714 map splits opinions. According to Inglis et al., Moll makes a definite attempt to improve the outline of Scotland and get away from: “the slavish copying of the out-of-date maps of the Gordon-Blaeu outline” (Inglis et al., 1934, p. 33). The Early Maps of Scotland agrees that the Moll map does move away from the Gordon-Blaeu outline, specifically: “The north coast has been improved and the far north-west corner extended to include Cape Wrath, even if still named Faro Head; the west coast is considerably straightened, and Arran is given its correct north-south axis instead of east-west as shown by Blaeu and Sanson” (The Early Maps of Scotland, 1973, pp. 84-85). The map is however still far from accurate, the Gordon-Blaeu placement of Skye and the Western Isles is maintained with some alterations, whilst the north-east region of Aberdeen and Banff is much too contracted. It is thought that Moll made no surveys of his own; rather he says in his county atlas of Scotland, that he made use of “Gordonius a Straloch, Tim. Pont, and John Adair, late Geographer for Scotland, who surveyed the coast”. The north coast of Scotland in Moll’s map is quite different from the Gordon-Blaeu outline, being based most likely of the maritime surveys of John Adair. Moll is however still reliant on the regional maps from the Blaeu atlas for place names, sometimes choosing a name of little importance. The National Library of Scotland (NLS, 2009 a) emphasises the influence of John Adair on Moll’s work, and that Moll went on to influence many later 18th century mapmakers. Withers (2005, p. 304) is of the opinion, that rather than improving on the Blaeu-Gordon outline, Moll actually diminishes it; replicating Blaeu rather than replacing him. While not commenting on Moll’s 1714 map, Stone (1972, p.1) describes Moll’s set of Thirty Six New and Correct Maps of Scotland (1725) as: “nothing more than crude reductions of Blaeu’s work”.

1745 – John ELPHINSTONE – A new & correct map of North Britain

Both Inglis et al. (1934, p. 35) and The Early Maps of Scotland (1973, p. 87) agree that John Elphinstone’s map of 1745 makes a great improvement on the outline of Scotland. According to Inglis et al. it is only let down by its “inability to put the Lochs from Inverness to Fort William in a straight line” (Inglis et al., 1934, p. 35) and the fact that “the Hebridies and Skye remain as in Blaeu’s Atlas” (Inglis et al., 1934, p. 35). The Early Maps of Scotland is more detailed in its analysis: “the north coast is based on Bryce’s map of 1744, the east coast from Berwick to Aberdeen on Adair’s manuscript maps. The Orkney Islands are based on the chart of Grenvile Collins. Elphinstone’s west coast shows some improvement on Moll” (1973, p. 87). The inaccuracies of Skye and the Hebrides are blamed on the lack of a proper survey of the islands. Elphinstone has left in faint traces of where the outlines of Moll (1714) and John Senex (1721) differ from his own map as comparison for his own improvements (Arrowsmith, 1809).

1751 – James Dorret – A correct map of Scotland from new surveys

Inglis et al. describe Dorret’s 1751 map as “an immense improvement on any-thing that preceded it” (Inglis et al., 1934, p. 38) and it became the basis of practically all the maps until Arrowsmith published his map in 1807. The map is only let down by the curve in the Great Glen and Skye and Islay not being well done. According to The Early Maps of Scotland (1973, pp, 91-92) even Skye and Lewis are shown with some degree of accuracy (although not perfect as later in the book it is said that: “Skye and Islay are not well done” (Early Maps of Scotland, 1973, p. 188)), for the first time Lewis no longer has the flat top characteristic of the Gordon-Blaeu outline. Lewis is based on the maritime charts of Murdoch Mackenzie published in 1750; this is evident because “Mackenzie’s survey came no further south than Loch Resort with the result that Dorret’s map, and all the maps based on Dorret, show a large bay in the place of the western half of North Harris” (Early Maps of Scotland, 1973, pp. 91-92). The shortening of northern Scotland present in Elphinstone has been corrected along with improvements on the shape of North and South Uist, Mull, Islay and Kintyre. Strangely there is still a bend in the Great Glen, despite the sources of information Dorret had access to. According to Tooley (1978, p.89), Dorret’s map was only the standard map for thirty years, until the publication of John Ainsley’s map in 1789. Withers (2005, p. 304) only criticises Skye and the Great Glen. The National Library of Scotland (NLS, 2009 a) suggests that Dorret may have received new information from Roy’s Military Survey, in addition to estate and marine surveys.

Sources:

Andrews, J. H., The Maps of the Escheated Counties of Ulster, Proceedings of the Royal Irish Academy 74, C, 1974

Arrowsmith, A,. Memoir relative to the Construction of the Map of Scotland published by Aaron Arrowsmith in the year 1807, London: Arrowsmith, 1809

Balletti, C,. Georeference in the analysis of the geometric content of early maps, e-Perimetron 1, No. 1, 2006

Blakemore, M. J. and J. B. Harley., Concepts in the history of cartography. A review and perspective, Cartographica 17, 1980

Boutoura, C. and E. Livieratos, Some fundamentals for the study of the geometry of early maps by comparative methods, e-Perimetron 1, No. 1, 2006

Cash, C. G., The first topographical survey of Scotland, Scottish Geographical Magazine 17, 1901

Cash, C. G., Manuscript maps by Pont, the Gordons, and Adair, in the Advocates’ Library Edinburgh, Scottish Geographical Magazine 23, 1907

Fleet, C., The Blaeu Atlas of Scotland: Background and introduction, Scottish Geographical Journal 121, Issue 3, 2005

Hardy, R.L., Multiquadratic Equations of Topography and Other Irregular Surfaces, Journal of Geophysical Research 76, No. 8, 1971

Hodson, D, “Herman Moll: Scotland” in Country Atlases of the British Isles Published after 1703: a bibliography. Vol 1: Atlases published 1704 to 1742 and their subsequent editions, Welwyn: Tewin Press, 1984

Hu, B., Assessing the Accuracy of The Map of the Prefectural Capital of 1261 Using Geographic Information Systems, The Professional Geographer 53, 2001

Inglis, H. G., J. Mathieson, and C. B. B. Watson, The Early Maps of Scotland with an account of the Ordnance Survey, Edinburgh: Royal Scottish Geographical Society, 1934

Jenny, B., MapAnalyst – A digital tool for the analysis of the planimetric accuracy of historical maps, e-Perimetron 1, No. 3, 2006

Jenny, B., Weber, A., and L. Hurni, Visualising the planimetric accuracy of historical maps with MapAnalyst, Cartographica 42, Vol. 1, 2007

Koeman, C., JOAN BLAEU and his Grand Atlas, Amsterdam: Theatrvm orbis terrarvm LTD, 1970

Lilley, K. D., C. D Lloyd and B. M. S Campbell, Mapping the Realm: A New Look at the Gough Map of Britain (c.1360), Imago Mundi 61, No1, 2009

Lloyd, C. and K. Lilley, Cartographic veracity in medieval mapping: Analyzing geographical variation in the Gough Map of Great Britain, Annals of the Association of American Geographers 99, No.1, 2009

Moore, J., Scottish cartography in the later Stuart era, 1660-1714, Scottish Tradition 14, 1986-87

Moore, J., The Historical Cartography of Scotland, 2nd ed., Aberdeen: University of Aberdeen, 1991

Moore, J., The Blaeu atlas of Scotland: Scotland from Theatrum Orbis Terrarum sive Atlas Novus pars quinta published by Joan Blaeu, Amsterdam, 1654, Scottish Geographical Journal 123, Vol. 2, 2007

NLS a (The National Library of Scotland), Maps of Scotland 1560-1928, http://www.nls.uk/maps/scotland/info.html, 2009, accessed 19.11.09

NLS b (The National Library of Scotland), Technical Information, http://www.nls.uk/pont/technical.html, 2009, accessed 19.11.09

Park, C., “geographical information system” in A Dictionary of Environment and Conservation, Oxford Reference Online, Oxford University Press, 2007, http://www.oxfordreference.com/views/ENTRY.html?subview=Main&entry=t244.e3319, accessed 24.11.09

Pearson, B. C., Comparative Accuracy in Four Civil War Maps of the Shenandoah Valley: A GIS Analysis, The Professional Geographer 57, No. 3, 2005

Royal Scottish Geographical Society, The Early Maps of Scotland to 1850, Vol 1 (With “A History of Scottish Maps” by D.G. Moir), 3rd ed., Edinburgh: Royal Scottish Geographical Society, 1983

Stone, J .C., The preparation of the Blaeu maps of Scotland: a further assessment, Scottish Geographical Magazine 86, 1970

Stone, J .C., Maps of Scotland by Timothy Pont, Robert and James Gordon and Joan Blaeu: A Study in Historical Cartography. Unpublished PhD thesis, Department of Geography, University of Aberdeen, 1972

Stone, J. C., “The study of the early maps of Scotland – the way ahead” in Cartography: the way ahead, ed. Wood Michael, Norwich: Geo Books, 1987

Stone, J. C., The Pont Manuscript Maps of Scotland: sixteenth century origins of a Blaeu atlas, Tring: Map Collector, 1989

Stone, J. C. and A. M. D. Gemmell, An experiment in the comparative analysis of distortion on historical maps, The Cartographic Journal 14, 1977

Tooley, R. V., Maps and map-makers, 6th ed., London: Batsford, 1978

Van Der Krogt, P., The place of the ‘atlas of Scotland’ in the atlas production of Willem Jansz and Joan Blaeu, Scottish Geographical Journal 121, Issue 3, 2005

Withers, C. W. J., The social nature of map making in the Scottish enlightenment c.1682-c.1832, Imago Mundi 54, No. 1, 2002

Withers, C. W. J., Working with old maps: Charting the reception and legacy of Blaeu’s 1654 Atlas Novus, Scottish Geographical Journal 121, Issue 3, 2005

Loading CSV XY Data into PostGIS, with Code-Point Open

Now that we have downloaded some OS OpenData, we can look into working with it.

Loading .csv data into PostGIS was surprisingly difficult, but can be done in a number of ways. This post will cover doing it using a .vrt (virtual raster) file.

In this example we are using Ordnance Survey and ultimately Royal Mail: Code-Point Open data. This can be found from OpenData and is a CSV point file for UK postcodes.

First we need to modify the header file provided with Code-Point:

Screenshot from 2013-10-09 20:21:32

I have deleted the short codes and replaced them with sensible headers. You should only have one line in your header file.

Firstly the files are provided split up into a number of different files. We can combine all of the .csv files using the cat command in Linux.

So this will combine all of the individual .csv files into the codepoint.csv file. Your header file should be the top line.

So now we can look at the data using ogrinfo, part of the gdal suite.

We create a virtual raster definition file with our X and Y columns, and projection defined:

So now we can run “ogrinfo” on codepoint.vrt:

Result:

Screenshot from 2013-10-09 21:25:42

So we can see that there was an error opening the codepoint.csv file itself but the .vrt worked fine. This is probably down to a memory issue on my part. So your mileage may vary. I tried again with just one postcode area and it worked fine (ab postcode area .csv was named codepoint.csv):

Screenshot from 2013-10-09 21:24:35

So I need to find a more memory friendly way to do this. So I will load the postcode files one at a time, appending them to any data that is already loaded.

So first I need to add the header row to the top of each file. The cat command worked really well last time so lets try that.

Rename the header file to .txt. So it will be called Code-Point_Open_Column_Headers.txt and consists of only one row with our desired column headers.

Write a bash script that adds the header file to the beginning of each .csv file:

Run the file:

Delete all of the .csv file (not the .csv.csv files!)

And we rename .csv.csv to .csv:

Then we can create the .vrt files for each csv file:

So we want to create the equivalent of for each csv file:

So we can do this using another bash script:

And finally we can load the files using a final bash script (this could be done by looping through the files as well):

So the ogr2ogr command is:

ogr2ogr -nlt PROMOTE_TO_MULTI -progress -nln codepoint -skipfailures -lco PRECISION=no -f PostgreSQL PG:”dbname=’os_open’ host=’localhost’ port=’5432′ user=’os_open’ password=’os_open’” source_file

Explained:

-nlt | PROMOTE_TO_MULTI, creates it as a multipart geometry, not important for points

-progress | Shows a progress bar

-nln codepoint | The name of the layer in the database, so because we are appending data to the first csv file loaded this is important.

-skipfailures -lco PRECISION=no | Personal preference

-f PostgreSQL PG:”dbname=’os_open’ host=’localhost’ port=’5432′ user=’os_open’ password=’os_open’” | Destination, so the details of your PostGIS database (see setting up PostGIS for help).

The latter commands also have the -append tag, which means they will be appended to the first one loaded.

Excellent. Though this was a bit intensive. Doing this in QGIS would have been a lot easier. However this can now be scripted and automatically run just by replacing the original .csv input files allowing for an easy update of the data when a new Cope-Point dataset is released.

Sources:

http://www.debian-administration.org/articles/150

Downloading OS OpenData (GIS) in Bulk

The Ordnance Survey have begun to release their data as free to use. I think this is great, and despite what some say, I think the selection is good as well. I can see why MasterMap and AddressBase, their flagship products still cost money, there are some significant costs associated with producing them.

There is however still a bit of a barrier of entry, into using OS OpenData. This guide will hopefully make downloading the data a little bit easier.

OS OpenData needs to be ordered from:

https://www.ordnancesurvey.co.uk/opendatadownload/products.html

There is no cost, but a valid e-mail address is required.

For my purposes I started with OS Street View® and OS VectorMap™ District for the whole country.

You will receive an e-mail with the download links for the data. Unfortunately these are individual links since the data is split up into tiles. 55 tiles in total for the whole of the UK, so we want some way to automate the download of these links.

In comes curl, made by some nice Swedes. curl is a command line tool which will download the link provided.

Attempt number 1:

Result:

Not quite there, missing the quotes:

Which was a success, but it started downloading the file into the terminal (ctrl+c stopped it):

Screenshot from 2013-10-03 20:16:12

So we need an output file as well:

Success:

Screenshot from 2013-10-04 08:43:17

To run this in bulk, just format the commands in LibreOffice (Excel) and create a simple shell script.

Copy the commands into a file and then:

To load the downloaded data into PostGIS, see my previous post on loading data into PostGIS.

Also a few tips for newer users of OS OpenData:

They are all in the British National Grid Projection (EPSG:27700). Watch out for this in QGIS. Version 1.8 used the 3 parameter conversion while 2.0 uses the 9 parameter. This means that if you created data in QGIS 1.8 and are now using 2.0, your data may not line up if you have “on the fly” CRS transformation turned on. To fix this just go into the layer properties and re-define the projection as British National Grid (EPSG:27700) for all the layers.

If you are working with Raster data you need to copy the .tfw files into the same directory as the .tif files. The .tfw (tiff world files) tell the .tif image where they should be in the world.

Georeferencing vector data using QGIS and ogr2ogr

This post will cover:

  1. Digitising rasters using QGIS.
  2. Loading data to PostGIS using SPIT.
  3. Reprojecting raster data using QGIS.
  4. Lines within a polygon selection in PostGIS.
  5. Georeferencing vector files using ogr2ogr, search for “The georeferencing:” to skip the other steps.

Georeferencing vector data has long been very difficult using the open-source GIS stack. That has all changed with the release of GDAL 1.10.0. Now georeferencing can be done using the vector translator ogr2ogr.

About half a decade ago I wrote my undergraduate dissertation on: “The influence of the Blaeu Atlas of Scotland on subsequent maps of Scotland”. While I was very proud of the work at the time, and the grade was good, the markers comments could be paraphrased as: A very good literature review on the Blaeu Atlas, however somewhat weak on analysis”. After a Master’s degree I could not agree more. At the time I lacked the knowledge and experience of working with projections. The release of GDAL 1.10.0 gives the perfect opportunity to return and correct those mistakes.

One of my favourite aspects of the Blaeu analysis was a comparison of the Blaeu 1654 Atlas of Scotland coastline to the coastline of subsequent maps of Scotland. I originally did this in GIMP and manually resized the outlines to close approximations. The end result looked good, but it lack real scientific rigour.

Original comparison:

1654_Blaeu_map_of_scotland_outline

A huge thanks to the National Library of Scotland for their work in digitising the Blaeu Atlas of Scotland. A great online version and overview can be found at:  http://maps.nls.uk/atlas/blaeu/

With the outline used today found at: http://maps.nls.uk/view/00000383

Originally the maps were provided as .jpeg files with a simple viewer used for zooming. This meant that editing the URL would allow one to retrieve a very large version of the of the map directly. Now the maps are served using a a very nice javascript map viewer and digital copies can be purchased.

First we need to create a vector outline from the raster image. This is a simple case of adding the .jpeg image in as a raster file. When prompted for a CRS I chose EPSG:4030 “Unknown datum based upon the WGS 84 ellipsoid” this is just used for meta-data at this point. What you choose does not matter. We just want a vector file in cartesian co-ordinates. We create a new shapefile layer and trace outline. My co-ordinate capture will be very rough. If you are working with more modern maps, you should be as careful as required:

Screenshot from 2014-01-20 20:43:20 

Whenever I needed a break I would end the line. Then I could start a new line snapped to the end of the last one. In the end you can select all of the line segments and “merge selected features” from the advanced digitising toolbar. This would likely have been enough, but I was concerned there would be two overlapping nodes and that I would have a multi-part line. So I exploded the lines into points using the Vector>Geometry Tools>Extract nodes tool. Then Using the handy Points2One plugin (thanks Pavol Kapusta) I could stitch the nodes back together into a single line. If you’re going to do something might as well do it correctly. In the end we have 5116 nodes:

Screenshot from 2014-01-20 23:45:09

With the Blaeu outline captured we need a basemap to georeference it to. In the end we want to re-projectiong it into the WGS 84 / World Mercator projection (EPSG:3395). Mercator will provide a good base of what the original mapper would have been trying to capture. In the UK we can turn to the Ordnance Survey for a basemap, however that basemap needs to be reprojected to EPSG:3395. This is simply done in within QGIS by selecting the “Enable on-the-fly CRS transformation” to EPSG:3395 from Project>project Properties>CRS tab.

From the Ordnance Survey (OS) we have a few open datasets to choose from:

  • OS BoundaryLine (High Water Polyline)
  • OS BoundaryLine (european electoral regions)

We can see how they line up with the raster products:

  • OS MiniScale
  • OS 250K

We can see that the High Water Polyline follows MiniScale much better than the electoral regions:

Screenshot from 2014-01-21 20:13:23Screenshot from 2014-01-21 20:13:41

It doesn’t line up perfectly with the 250k raster (250k is probably too accurate for this study):

Screenshot from 2014-01-21 20:19:07

But on the whole it will serve for this purpose, in addition Scotland can be easily extracted:

Screenshot from 2014-01-21 21:12:47Screenshot from 2014-01-21 21:04:42

Now we have the High Water Polyline, which looks good, but we only want mainland Scotland. Easy right?

Well… slighlty more complex. Unfortunately even the Scottish high water lines consisted of 16030 line segments. Select by location in QGIS using the European Electoral Regions (polygon) took too long (hours rather than minutes). So I had to resort to PostGIS to do the selection.

SPIT to PostGIS:

Screenshot from 2014-01-21 21:51:16

For setting up a PostGIS database see my previous post: Setting up PostgreSQL and PostGIS on Linux Mint.

The query in PostGIS/PostgreSQL to select all of the lines within a polygon (we use the OS boundary line electoral regions):

Result:

Query returned successfully: 4994 rows affected, 764779 ms execution time.

The 12 minute run time says a lot about my poor computer.

Then the lines are joined together, the geometries simplified, and multipart converted to singlepart. This allows us to get rid of the final islands. With the result back in QGIS:

Screenshot from 2014-01-22 18:03:40

Closeup:

Screenshot from 2014-01-23 00:04:17

Final result projected into WGS 84 / World Mercator projection (EPSG:3395), which really shows the distortion caused by the British National Grid (EPSG:27700):

Screenshot from 2014-01-23 00:12:45

We now have a cartesian vector file ready for georeferencing, an OS raster background map to use for georeferencing (OS MiniScale) and an OS vector outline BoundaryLine (High Water Polyline) to use for the final comparison.

The georeferencing:

We open up two QGIS projects. In one we have our cartesian vector file. In the other we have our basemap:

Screenshot from 2014-01-23 00:16:01

Using the co-ordinate capture tool, we capture the same point from both projects. Basically we are just capturing co-ordinate pairs from the two maps. The “Copy to clipboard” feature comes in very handy here.

Screenshot from 2014-01-21 18:32:34

These captured co-ordinates should be pasted into a text file. Something like this:

Of note here is that the co-ordinate capture tool captured the co-ordinate twice for me. How it works is that it captures one in the original CRS and the selected one. It is also important to keep your co-ordinates in order. I captured the cartesian co-ordinates on the first row and the WGS84 ones on the second row. Also I have attached memorable names to the points, in case there is an issue, the culprit point can be easily be identified.

For my project 23 points were captured:

Screenshot from 2014-01-23 00:48:57

After the points are captured we switch to ogr2ogr to perform the actual reprojection and to convert the OS outline into World Mercator.

Scotland Mainland Outline from Boundary Line High Water Mark to World Mercator (while I’m converting I will convert it to a GeoJSON for viewing in leaflet (this will be covered by a later post) and strip out any attributes to decrease the file size):

The key here is the -gcp tag. For each control point (co-ordinate pair) we have one tag:

Turns into:

So: -gcp ungeoref_x ungeoref_y georef_x georef_y

One -gcp tag for each point.

Means that it will use the a first order polynomial transformation, so affine. For myself the less distortion the better.

The end result is great, it really shows how advanced the Blaeu Atlas was for a 1654 map:

blaeu_os_comparison

If you are interested in buying a historically amazing map:

Blaeu Atlas Maior of 1665 - Including the atlas of Scotland.

Also please visit the NLS, without who this would have been impossible, they also have great changing exhibits:

National Library of Scotland

NLS Exhibitions

Loading Natural Earth data to PostGIS PostgreSQL

Natural Earth provides some of the best data for large scale mapping. It is clean, accurate, extensive, at a number of different scales, and best of all free.

To load the data it into PostGIS (PostgreSQL) we will use the vector tools provided in GDAL. Mainly ogr2ogr.

After downloading the data. I went for all of the vector data in ShapeFile format. First I need to generate a list of datasets and their respective file paths. This will be put into a spreadsheet and the command to load the data will be applied to each line, and finally it will be run using a shell script. Setting up a PostGIS database is covered in my previous post.

My Natural Earth data consisted of:

28 directories, 1472 files

So a little automation is needed. Interestingly there were also a few .gdb files “ne_10m_admin_1_label_points.gdb”. Those we can look into at a later date.

To begin:

Produced a decent result, but not quite what I was looking for.

Was much better, although with a bit more tuning I’m sure ls would have achieved a better result.

tree

After a bit of work in the spreadsheet, I had what I wanted. Perhaps not the most elegant solution, but certainly enough for my purposes.

Now for the ogr2ogr command:

Ogr2ogr is a file converter, which does so much more. In this case we are converting the ShapeFiles into tables in a PostGIS database. Essentially you want to copy the beginning part of the command in front of the files you want to load, changing only: “10m_cultural/ne_10m_admin_0_antarctic_claim_limit_lines.shp” .

Our settings:

-nlt PROMOTE_TO_MULTI | Loads all out files as if they were multi-part polygons. This means that a multi-part polygon wont fail the loading. This is a PostGIS requirement.

-progress | Shows a progress bar.

-skipfailures | Will not stop for each failure.

-overwrite | Overwrites a table if there is one with the same name. Our tables will be called whatever the ShaeFile is called since we are not specifying a name.

-lco PRECISION=no | Helps keep numbers manageable, especially with this data where precision is not important.

-f PostgreSQL PG:”dbname=’DatabaseName’ host=’IpAddressOfHost’ port=’5432′ user=’Username’ password=’Password’” | Details of the database where we are connecting to.

Now we are ready to run the commands. While ogr2ogr commands can be pasted straight into the terminal, for this task that is not really feasible. So we can create a simple shell script.

Copy the commands into a file and then:

Finally there was one final error, with ne_10m_populated_places.shp. This was due to encoding. The encoding for the ogr2ogr tool can be changed from UFT8 to LATIN1 using:

After which the file loaded swimmingly.

Now for some mapping.

Thanks to:

http://unix.stackexchange.com/questions/92387/pasting-into-terminal-deteriorating?noredirect=1#comment139520_92387

http://lists.osgeo.org/pipermail/gdal-dev/2009-May/020771.html