The Google Maps API and Custom Overlays

In our previous tutorial we looked at how you can add markers and customize the colors and menus of a Google map using the service's API. This tutorial takes things a step further, explaining how you can create your own custom map and overlay it onto a Google map (such as my 1896 map of Portsmouth).

In this tutorial we'll add a 'weather predictions' overlay to the map we built previously. The weather map I use is entirely fictitious, but the techniques covered will give you the freedom to create all sorts of interesting things! You'll be able to build historic maps, custom drawn maps, close ups of specific areas such as a university campus - the list is endless!

A few things you need to know before you start:

• A Google map is made using many image ‘tiles’ (see image above). In other words, the map is split into an invisible grid and each grid square is a separate image file of 256x256 pixels.
• You need a set of image tiles for each zoom level. The zoom level is a number, from zero upwards, which changes as you zoom in and out of the map (see image above).
• Your map overlay starts its life as a really big image (a jpeg or png), which you then ‘slice up' into tiles. It's a repetitive process, in the sense that the image is sliced up once to create a set of tiles for the first zoom level, and then it's sliced up again into smaller segments to create the second set of tiles for the next zoom level, and so on.

This tutorial explains each step, including the software tools that are (fortunately!) available to automate things and make life a lot easier. The source files for every step are available in the download package.

Downloading a Template

To get started, the first step is to make a template onto which you can create your own overlay.

Your aim is to create a template in Photoshop (or equivalent graphics application) which is exactly the same size and shape as the Google map you are going to overlay it on top of, when it's at its highest zoom level.  (maptiler.org is useful if you aren't sure what the highest zoom level you need is).

To do this, you can save a template map using maptiler's mashup and then enlarge it in Photoshop to your desired maximum zoom level. For example:

• Use your graphics application to open the template map image you've saved.
• Go to the Tiles à la Google Maps website and take a screenshot of a segment of your Google map, at your required maximum zoom level.
• Paste the screenshot into your Photoshop file.
• Then enlarge your template map image so it exactly aligns with the corresponding zoomed map segment.
• Next, delete the zoomed segment. The quality of your template map wont be great, but this doesn't matter as you just need it for the outline and you'll delete it eventually.

At this stage, I'm not going to complicate things! So in a slight change to the previous tutorial, I’ve changed the max zoom level from 12 to 10 to give us a manageable file size to work with.

Creating Your Own Map

Once you have your map jpg using the process described above, open it in Photoshop, or your preferred graphics application.

You'll use this image as a template and draw your own map on top of it. This is to ensure that your map perfectly lines up with the Google map.

• Lock your map template layer.
• Add a new layer/s.
• Create something amazing. (Actually, don’t get too carried away to start with. In fact, it's a good idea to create something basic, like my weather map, and follow the rest of the tutorial to get a feel for what’s happening. Then create something amazing!)
• Hide the map template layer.
• Save your map as a png or jpg (if you have transparent parts, such as the sea in this tutorial, then go for a png).

Take a look at the weather map image I created at this stage, which is available in the source download.

Slicing up Your Map

There are a couple of ways you can slice up your image to create the map tiles. If you want a bit of background info about what you are doing, read on. Otherwise, if you don't care why or how it works, just skip to the next section!

Projection Systems

A projection system, in the geographical sense, is how a flat map is created from a round object, i.e. the globe. There are all sorts of (frankly baffling) formulae and algorithms used to achieve this.  All you need to know is that different projection systems result in different flat maps of the world.

To create an overlap for a Google map you need to use the ‘Spherical Mercator’ system which uses the Mercator projection. This is defined as EPSG:900913 or EPSG:3857.  To find out more about projection systems see the Tiles à la Google Maps webpage.

Finding Each Side of Your Map

Before you can generate your tiles, you need to work out the position of the north, south, west and east sides of your map. As you are using the ‘Spherical Mercator’ system, these need to be in spherical meters as opposed to latitude or longitude.

You can find these position values using this mashup. I've found that the easiest method is to zoom right in and find one side at a time.  So find the value for the west side of your map, make a note of it, then find the value for the south side, and so on.

Once you've found these four values, you can now 'slice up' your map into tiles. There are a couple of options available.

Slicing Option 1: MapTiler

Happily, the MapTiler software is really straightforward to use!  At the time of writing this tutorial, MapTiler is updating their software. But regardless of which version you download, these are the basic steps:

• Download Maptiler. Or the newer Beta Version.
• Start the programme and select Google maps compatible option.
• Then add the map image you created in the previous step.
• You'll be asked to give the georeference of the boundaries of your map, i.e. the position in spherical meters of each side that you've just found.  In version 0.41 (the more recent of the two versions) this is referred to as a Bounding Box.  Be careful with the order you enter the values!
• When asked for your spacial reference / coordinate system use ESPG: 3857 . (If you are using the old version 1 (beta) version of MapTiler, then use EPSG:900913).
• Then select the Zoom Levels you want to generate tiles for. The max value should be the value you saved your template map at when you created it (above). The Tiles à la Google Maps page is useful to work out what you need for the min zoom value.
• Continue to the end of the wizard and start the Render. This can take a while; a good opportunity to get a coffee!

Note: If you are using the new version of MapTiler, there is a gotcha; your map tiles are watermarked. I'm guessing in some cases, perhaps for a Uni project or technical proof-of-concept, this isn't an issue anyway.  However, there are a couple of ways to get round this. The most obvious way is to buy the upgrade for $20 I believe. The other option is to use the GDAL2Tiles python script.  My personal preference is to go for the upgrade because I'd much rather spend my time on the creative stuff, rather than enduring the irritations of setting up the scripting environment, but you may feel differently.

Slicing Option 2: GDAL2Tiles

An alternative to MapTiler is the GDAL2Tiles script. The MapTiler software is actually built on top of this script, which runs through the command prompt interface.

GDAL2Tiles offers further possibilities but it's also a bit trickier to use; so only continue reading if you have a sense of adventure!

The first stage is setting up the environment to run the GDAL2Tiles script. How you do this depends on your operating system.  If you are using a 32bit Windows machine (click here if you are not sure), then you can use the can use OSGeo4W.

• You first need to install Python and check python is on your path variable.
• Then download OSGeo4W.
• Run the installer and select Advanced Install.
• Select Libs and select gdal and gdal-python in the select packages.
• Carry on and complete the installation.
• When it’s done, you should have the OSGeo4W icon on your desktop. This is a batch file to start the command  prompt.

If you are using a 64bit Windows machine (click here if you are not sure), then the setup is a bit different. These instructions are quoted (with kind permission) directly from Jaerock Kwon’s blog, who deserves full credit for giving such detailed instructions.

• Install Python from x86-64.
• Check python is on your path variable.
• Run python.exe. (e.g. click on C:\Python27\python.py) We have to find out the compiler version that built the python. In my case, the Python version is 2.7.5 and it was compiled and built with Python 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)] on win32.
• GDAL binary packages for 64bit machines can be found at http://vbkto.dyndns.org/sdk/. Select a corresponding version in the table. In my case,  release-1500-x64-gdal-1-10-mapserver-6-2 is the right version in the "MSVC2008 (Win64) -stable” row because the Python was built by 1500.
• Download Generic installer for the GDAL core components. In my case - gdal-110-1500-x64-core.msi
• Download the installer for the GDAL python bindings (requires to install the GDAL core). In my case GDAL-1.10.0.win-amd64-py2.7.msi - I chose this because 1.10.0 is the latest and my Python is 2.7.5.
• Install the GDAL core components. There is no option to choose the destination folder for GDAL core. It will be installed into the “C:\Program Files\GDAL” folder.
• Install the GDAL python bindings.
• After the binding, you may move GDAL folder in C:\Program Files into wherever you want to.
• Finally, click on the Start menu and then All Programs and then GDAL. This should start the command command line prompt.

So assuming you’ve got through the (sometimes traumatic!) process of setting up the GDAL2Tiles software environment, it's time to do something with it.

Whether you've got the OSGeo4W or the GDAL environment, your command prompt should look similar to the above screen shot. You can now start using the GDAL2Tiles script.

First, you need to navigate into the folder where the map you want to slice up is saved. (These commands are useful to get where you need to be: dir gives you a list of everything in the current directory, cd means change directory, cd .. means move one level up the tree).

Once you are in the correct folder, you can start creating your tiles. To create the demo map in this tutorial, these are the commands I used -

gdal_translate -of VRT -a_srs EPSG:900913 -gcp 0 0 -910307.6791052371 8109796.717743561 -gcp 7248 0 196252.07012218982 8109796.717743561 -gcp 7248 10929 196252.07012218982 6438749.514630105 part2_rainfall_map.png out.vrt

gdal2tiles.py -p mercator -z 6-10 out.vrt

To explain what these commands are doing, see the gdal_translate page and the gdal2tiles.py  page.  There is also a GDAL warp command that’s closely related, but it’s not needed in this case. A few things to note are:

• The three -gcp properties relate to three corners of your map. The first pair of values after each gcp property relates to the dimensions in pixels of your input image file, and the second pair of values are the coordinates in spherical meters (not latitude and longitude - see above).

So to apply this to your map:

gdal_translate -of VRT -a_srs EPSG:900913 -gcp 0 0 west north -gcp width_of_your_input_map_file_in_pixels 0 east north -gcp width_of_your_input_map_file_in_pixels height_of_your_input_map_file_in_pixels east south your_input_file_name.png your_output_file_name.vrt

gdal2tiles.py -p mercator -z zoom_range name_of_output_file.vrt

After firing the second command, your computer will spring into life and start cutting up your image to create the map tiles. You may like to take a look at the exact commands I typed into the OSGeo4W  and the GDAL environments if you are unsure about navigating around folders using the command prompt.

Understanding the Output

Regardless of the method you used to generate your tiles, the folders generated will be the same.

All tiles on a Google map have an 'x' and 'y' value. (Yes - yet another different coordinate system!) The first level of folders are the zoom level, then inside that the next level of folders are the 'x' coordinate, and inside that the file name is the 'y' coordinate.

If you take a look at the map on this website you should see that the names of your tiles match up. In other words, go to your desired location, adjust the zoom level so that it matches your folder name, then look at the Google 'x' and 'y' values. You should see that they match your subfolder and file names for the corresponding map tiles.

To double check your tiles are OK, I strongly recommend copying one or two of the tiles generated for the lowest zoom level back into your map file (e.g. the Photoshop file) created during the previous stage. Check that they fit exactly on top of your map. Take a look in the source files I've provided for an example of this.

If all is well then - congratulations - the hard work is done!

Help, My Map Tiles Aren't Named Correctly!

Oh dear. This will happen if you used the older version of the MapTiler sofware or the GDAL2Tiles script. Your files (but not the folders) are named using the TMS naming convention.  If you look at the tiles on this page you can see that each tile has two x and y values; ie. a Google pair and a TMS pair.  We just need to rename them to the Google map naming convention.

This renaming can be done manually, but it will take far too long for most maps. Fortunately, there's a handy shell script that will do the job for you. You need to know a bit about very basic Linux commands, such as how to change folder. Before running the shell script, make a copy of your tiles to work on.

1

#!/bin/sh

2

for thisPath in `ls -d */*/*`

3

do

4

thisFile=${thisPath#*/*/}

5

oldY=${thisFile%.png}

6

zoomX=${thisPath%/*}

7

zoom=${thisPath%/*/*}

8

newY=$(((1<<zoom) - oldY - 1))

9

mv ${zoomX}/${oldY}.png ${zoomX}/${newY}.png

10

done

Copy and paste the above script into Notepad and save it with the name 'rename' and the '.sh' extension.  When saving the ‘sh’ file in notepad, make sure ‘all files' next to ‘save as type’ is selected, and put the name in quotes e.g. “rename.sh”.

If you are using a Mac, this shell script can be run in Terminal (select Applications then Utilities then Terminal).

If you are on a Windows machine, then to run shell scripts (or ‘.sh’ files) you need to use Cygwin, which is a Linux like environment for Windows:

• Download and install Cygwin
• Copy the 'rename.sh' file you've just saved into the cygwin\bin folder.
• Start Cygwin.
• Move to the cygwin\bin folder. (ie.  cd means change directory, cd ..  means move up the tree one level, and ls means show the files and folders in the current directory.)
• Make rename.sh executable by running chmod 755 rename.sh
• You also need to convert it into a Linux file before running it otherwise you get ‘\r’ errors caused by the line breaks. So run make sure you are still in the cygwin\bin folder and run d2u rename.sh
• Then move to the folder that contains your tiles (See the screen shot above).
• Then to run the script, type rename.sh

After the shell script has run, you can check it's done the job by taking another look at your files. Check whether their names now match the Google coordinates on this page.

If you compare the file names in the ‘Part 3 files’ and the ‘Part 4 files’ folders in the source files, you can see the difference.

Compressing Your Map Titles

To make sure your map runs as smoothly as possible, you can compress your map tiles.

There are various free compression utilities for png images, but I struggled (and failed) to find one that was completely free and did the job. For example, the popular PNGGauntlet software didn't preserve the folder structure, whereas other free utilities had a limit on the maximum number of files.

In the end I had to bite the bullet and buy the PNGOUTWin software. This cost around £10, which is a bargain given the faffing around that it saved! PNGOUTWin both preserves the file structure and lets you compress all your files at once.

If you use PNGOUTWin, it’s pretty intuitive. My only word of warning is that you should make a copy of your tiles and then run the software on the copy. Also, beware that the software literally starts as soon as you have selected the folder!

JavaScript Magic to Bring it All Together

Okay, we’re almost there. All that remains is the small matter of attaching your lovely new map tiles to your Google map.

The Basic Code

First, take a look at a simple example of a tiled map overlay (source files are available from the link at the top of this page).

If you zoom in and out, the map overlay stays put.

1

//setting up the rainfall map overlay

2

var rainMapOverlay = new google.maps.ImageMapType({

3

 getTileUrl: function(coord, zoom) {

4

 return 'tiles/rainfall' + '/' +zoom+ '/' +coord.x+ '/' + coord.y +'.png';

5

 },

6

 tileSize: new google.maps.Size(256, 256)

7

});

This snippet is using the ImageMapType class to create the custom overlay. The coord.x relates to the folder name and the coord.y relates to the file name. These are joined together in the code to create the path to each tile.

Hooking it up to the UK Festival Map

This is the final product you are aiming for. To create this, you can use the same techniques explained near the end of the previous tutorial.

We first create a variable (rainfallOverlayToggle) to hold whether or not the weather overlay is currently showing. And, then creating a new option in the handelRequests function to deal with the situation when the umbrella button (brolly) is clicked.

1

//Displays the rainfall map when the unbrella button is clicked, and removed it if is already showing.

2

else if (buttonPressed === "rainfall"){

3

//If the rainfall map is NOT showing aleady then show it ...

4

if (rainfallOverlayToggle === 0){

5

//Overlays the rainfall map on top of the Google map

6

festivalMap.overlayMapTypes.insertAt(0, rainMapOverlay);

7

//Show the weather key.

8

festivalMap.controls[google.maps.ControlPosition.TOP_LEFT].push(weatherKeyDiv);

9

rainfallOverlayToggle = 1;

10

}

11

//If the rainfall map is showing already then hide it ...

12

else {

13

//remove the overlay map.

14

festivalMap.overlayMapTypes.removeAt(0, rainMapOverlay);

15

//remove the weather key

16

festivalMap.controls[google.maps.ControlPosition.TOP_LEFT].pop(weatherKeyDiv);

17

rainfallOverlayToggle = 0;

18

}

19

}

What Next?

Well done for reaching the end! If you have any questions at all, please ask away in the comments. The next tutorial will take a look at connecting to the Flickr API, allowing you to automatically grab photos from Flickr and display them on your map.

Image Credits

• Mercator projection map - Wikipedia
• Gall-Peters projection map - Wikipedia