Geospatial Power Tools Reviews [Book]

Thinking of buying my latest book?  We’ve finally got a few reviews on Amazon that might help you decide.  See my other post for more about the book.  Buy the PDF on Locate Press.com.

Reader Reviews

Geospatial Power Tools book cover
From Amazon.com

5.0 out of 5 stars This book makes a great reference manual for using GDAL/OGR suite of command line …,
January 24, 2015 By Leo Hsu
“The GDAL Toolkit is chuckful of ETL commandline tools for working with 100s of spatial (and not so spatial data sources). Sadly the GDAL website only provides the basic API command switches with very few examples to get a user going with. I was really excited when this book was announced and purchased as soon as it came out. This book makes a great reference manual for using GDAL/OGR suite of command line utilities.
Several chapters are devoted to each commandline tool, explaining what its for, the switches it has, and several examples of how to use each one. You’ll learn how to work with both vector/(basic data no vector) data sources and how to convert from one vector format to another. You’ll also learn how to work with raster data and how to transform from one raster data source to another as well as various operations you can perform on these.”

 

Continue reading Geospatial Power Tools Reviews [Book]

Converting Decimal Degree Coordinates

Converting Decimal Degree Coordinates to/from DMS Degrees Minutes Seconds

cs2cs command from GDAL/OGR toolset (gdal.org) - allows robust coordinate transformations.
cs2cs command from GDAL/OGR toolset (gdal.org) – allows robust coordinate transformations.

If you have files or apps that have to filter or convert coordinates – then the cs2cs command is for you.  It comes with most distributions of the GDAL/OGR (gdal.org) toolset.  Here is one popular example for converting between degrees minutes and seconds (DMS) and decimal degrees (DD).


Geospatial Power Tools book coverThe following is an excerpt from the book: Geospatial Power Tools – Open Source GDAL/OGR Command Line Tools by me, Tyler Mitchell.  The book is a comprehensive manual as well as a guide to typical data processing workflows, such as the following short sample…


Input coordinates can come from the command line or an external file. Assuming a file containing DMS (degree, minute, seconds) style, looks like:

124d10'20"W 52d14'22"N
122d20'05"W 54d12'00"N

Use the cs2cs command, specifying how the print format will be returned, using the -f option. In this case -f “%.6f”
is explicitly requesting a decimal degree number with 6 decimals:

cs2cs -f "%.6f" +proj=latlong +datum=WGS84 input.txt

Example Converting DMS to/from DD

This will return the results, notice no 3D/Z value was provided, so none is returned:

-124.172222 52.239444 0.000000
-122.334722 54.200000 0.000000

To do the inverse, remove the formatting option and provide a list of values in decimal degree (DD):

cs2cs +proj=latlong +datum=WGS84 inputdms.txt
124d10'19.999"W 52d14'21.998"N 0.000
122d20'4.999"W 54d12'N 0.000


Geospatial Power Tools is 350+ pages long – 100 of those pages cover these kinds of workflow topic examples. Each copy includes a complete (edited!) set of the GDAL/OGR command line documentation as well as the following topics/examples:

Workflow Table of Contents

  1. Report Raster Information – gdalinfo
  2. Web Services – Retrieving Rasters (WMS)
  3. Report Vector Information – ogrinfo
  4. Web Services – Retrieving Vectors (WFS)
  5. Translate Rasters – gdal_translate
  6. Translate Vectors – ogr2ogr
  7. Transform Rasters – gdalwarp
  8. Create Raster Overviews – gdaladdo
  9. Create Tile Map Structure – gdal2tiles
  10. MapServer Raster Tileindex – gdaltindex
  11. MapServer Vector Tileindex – ogrtindex
  12. Virtual Raster Format – gdalbuildvrt
  13. Virtual Vector Format – ogr2vrt
  14. Raster Mosaics – gdal_merge

Create Tile Map Structure – gdal2tiles command

Tiles in a Tile Map Server (TMS) context are basically raster map data that’s broken into tiny pre-rendered tiles for maximum web client loading efficiency. GDAL, with Python, can chop up your input raster into the folder/file name and numbering structures that TMS compliant clients expect.

OpenLayers mapping application showing natural earth dataset
Default OpenLayers application produced by the gdal2tiles command and a Natural Earth background dataset as input.

This is an excerpt from the book: Geospatial Power Tools – Open Source GDAL/OGR Command Line Tools by me, Tyler Mitchell.  The book is a comprehensive manual as well as a guide to typical data processing workflows, such as the following short sample…

The bonus with this utility is that it also creates a basic web mapping application that you can start using right away.

The script is designed to use georeferenced rasters, however, any raster should also work with the right options. The (georeferenced) Natural Earth raster dataset is used in the first examples, with a non-georeferenced raster at the end.

There are many options to tweak the output and setup of the map services; see the complete gdal2tiles chapter for more information.

Minimal TMS Generation

At the bare minimum an input file is needed:

gdal2tiles.py NE1_50M_SR_W.tif
Generating Base Tiles:
0...10...20...30...40...50...60...70...80...90...100 - done.
Generating Overview Tiles:
0...10...20...30...40...50...60...70...80...90...100 - done.

The output created is the same name as the input file, and include an array of sub-folders and sample web pages:

NE1_50M_SR_W
NE1_50M_SR_W/0
NE1_50M_SR_W/0/0
NE1_50M_SR_W/0/0/0.png
NE1_50M_SR_W/1
...
NE1_50M_SR_W/4/9/7.png
NE1_50M_SR_W/4/9/8.png
NE1_50M_SR_W/4/9/9.png
NE1_50M_SR_W/googlemaps.html
NE1_50M_SR_W/openlayers.html
NE1_50M_SR_W/tilemapresource.xml

Open the openlayers.html file in a web browser to see the results.

The default map loads a Google Maps layer, it will complain that you do not have an appropriate API key setup in the file, ignore it and switch to the OpenStreetMap layer in the right hand layer listing.

 

The resulting map should show your nicely coloured world map image from the Natural Earth dataset. The TMS Overlay option will show in the layer listing, so you can toggle it on/off to see that it truly is loading. Figure 5.2 (above) shows the result of our gdal2tiles command.


Geospatial Power Tools is 350+ pages long – 100 of those pages cover these kinds of workflow topic examples.  Each copy includes a complete (edited!) set of the GDAL/OGR command line documentation as well as the following topics/examples:

Workflow Table of Contents

  1. Report Raster Information – gdalinfo 23
  2. Web Services – Retrieving Rasters (WMS) 29
  3. Report Vector Information – ogrinfo 35
  4. Web Services – Retrieving Vectors (WFS) 45
  5. Translate Rasters – gdal_translate 49
  6. Translate Vectors – ogr2ogr 63
  7. Transform Rasters – gdalwarp 71
  8. Create Raster Overviews – gdaladdo 75
  9. Create Tile Map Structure – gdal2tiles 79
  10. MapServer Raster Tileindex – gdaltindex 85
  11. MapServer Vector Tileindex – ogrtindex 89
  12. Virtual Raster Format – gdalbuildvrt 93
  13. Virtual Vector Format – ogr2vrt 97
  14. Raster Mosaics – gdal_merge 107

Create a Union VRT from a folder of Vector files

The following is an excerpt from the book: Geospatial Power Tools – Open Source GDAL/OGR Command Line Tools by me, Tyler Mitchell.  The book is a comprehensive manual as well as a guide to typical data processing workflows, such as the following short sample…

The real power of VRT files comes into play when you want create virtual representations of features as well.  In this case, you can virtually tile together many individual layers as one.  At the present time you cannot do this with a single command line but it only takes adding two simple lines to the VRT XML file to make it start working.

Here we want to create a virtual vector layer from all the files containing lines in the ne/10m_cultural folder.

First, to keep it simple, create a folder and copy in only the files we are interested in:

mkdir ne/all_lines 
cp ne/10m_cultural/*lines* ne/all_lines

Then we can create our VRT file using ogr2vrt as shown in the previous example:

python ogr2vrt.py -relative ne/all_lines all_lines.vrt

If added to QGIS at this point, it will merely present a list of four layers to select to load. This is not what we want.

So next we edit the resulting all_lines.vrt file and add a line that tells GDAL/OGR that the contents are to be presented as a unioned layer with a given name (i.e. “UnionedLines”).

The added line is the second one below, along with the closing line second from the end:

<OGRVRTDataSource>
 <OGRVRTUnionLayer name="UnionedLines">
  <OGRVRTLayer name="ne_10m_admin_0_boundary_lines_disputed_areas">
   <SrcDataSource relativeToVRT="1" shared="1">
   ...
   <Field name="note" type="String" src="note" width="200"/>
  </OGRVRTLayer>
 </OGRVRTUnionLayer>
</OGRVRTDataSource>

Now loading it into QGIS automatically loads it as a single layer but, behind the scenes, it is a virtual representation of all four source layers.

On the map in Figure 5.8 the unionedLines layer is drawn on top using red lines, whereas all the source files (that I manually loaded) are shown with a light shading. This shows that the new virtual layer covers all the source layer features.

Unioned OGR VRT layers - source layers beneath final resulting merged layer
Unioned OGR VRT layers – source layers beneath final resulting merged layer

 


Geospatial Power Tools is 350 pages long – 100 of those pages cover these kinds of workflow topic examples.  Each copy includes a complete (edited!) set of the GDAL/OGR command line documentation as well as the following topics/examples:

Workflow Table of Contents

  1. Report Raster Information – gdalinfo 23
  2. Web Services – Retrieving Rasters (WMS) 29
  3. Report Vector Information – ogrinfo 35
  4. Web Services – Retrieving Vectors (WFS) 45
  5. Translate Rasters – gdal_translate 49
  6. Translate Vectors – ogr2ogr 63
  7. Transform Rasters – gdalwarp 71
  8. Create Raster Overviews – gdaladdo 75
  9. Create Tile Map Structure – gdal2tiles 79
  10. MapServer Raster Tileindex – gdaltindex 85
  11. MapServer Vector Tileindex – ogrtindex 89
  12. Virtual Raster Format – gdalbuildvrt 93
  13. Virtual Vector Format – ogr2vrt 97
  14. Raster Mosaics – gdal_merge 107

Query Vector Data Using a WHERE Clause – ogrinfo

The following is an excerpt from the book: Geospatial Power Tools – Open Source GDAL/OGR Command Line Tools by Tyler Mitchell.  The book is a comprehensive manual as well as a guide to typical data processing workflows, such as the following short sample…

Use SQL Query Syntax with ogrinfo

Use a SQL-style -where clause option to return only the features that meet the expression. In this case, only return the populated places features that meet the criteria of having NAME = ’Shanghai’:

$ ogrinfo 10m_cultural ne_10m_populated_places -where "NAME = 'Shanghai'"

... 
Feature Count: 1 Extent: (-179.589979, -89.982894) - (179.383304, 82.483323) 
... 
OGRFeature(ne_10m_populated_places):6282
 SCALERANK (Integer) = 1 
 NATSCALE (Integer) = 300 
 LABELRANK (Integer) = 1 
 FEATURECLA (String) = Admin-1 capital 
 NAME (String) = Shanghai
... 
 CITYALT (String) = (null) 
 popDiff (Integer) = 1 
 popPerc (Real) = 1.00000000000 
 ls_gross (Integer) = 0 
 POINT (121.434558819820154 31.218398311228327)

Building on the above, you can also query across all available layers, using the -al option and removing the specific layer name. Keep the same -where syntax and it will try to use it on each layer. In cases where a layer does not have the specific attribute, it will tell you, but will continue to process the other layers:

   ERROR 1: 'NAME' not recognised as an available field.

NOTE: More recent versions of ogrinfo appear to not support this and will likely give FAILURE messages instead.


Geospatial Power Tools is 350 pages long – 100 of those pages cover these kinds of workflow topic examples.  Each copy includes a complete (edited!) set of the GDAL/OGR command line documentation as well as the following topics/examples:

Workflow Table of Contents

  1. Report Raster Information – gdalinfo 23
  2. Web Services – Retrieving Rasters (WMS) 29
  3. Report Vector Information – ogrinfo 35
  4. Web Services – Retrieving Vectors (WFS) 45
  5. Translate Rasters – gdal_translate 49
  6. Translate Vectors – ogr2ogr 63
  7. Transform Rasters – gdalwarp 71
  8. Create Raster Overviews – gdaladdo 75
  9. Create Tile Map Structure – gdal2tiles 79
  10. MapServer Raster Tileindex – gdaltindex 85
  11. MapServer Vector Tileindex – ogrtindex 89
  12. Virtual Raster Format – gdalbuildvrt 93
  13. Virtual Vector Format – ogr2vrt 97
  14. Raster Mosaics – gdal_merge 107
%d bloggers like this: