Archive for the ‘PHP’ Category

Geocube Geograph Clusters in Google Earth – part 2

Wednesday, July 15th, 2009

Now ready with the next version of the Geograph Google Earth Clusters layer mentioned yesterday.

Google Earth Layer Geograph Google Earth Clusters

It’s the same download link, if you already have a copy, just right click and select Refresh on the “Geograph Google Earth Clusters” feature.

This has a number of advancements:

  • Filterable – click the main title in the Places panel to open configuration options in the popup balloon. Includes options to filter by keyword or Geograph contributor. (Requires Google Earth version 5)
  • Cubes/Cloud – Geocubes offers two clustering options, can how choose between then on the same popup as above
  • Thumbnails – Now shows a image thumbnail in the placemark balloons.
  • Shows a few coarse clusters when not zoomed on the British Isles
  • Source code available! Download the GPL licenced PHP source code here

But its not all rosey – this version seems to suffer from an annoying bug, when first loads sometimes the layer ‘freezes’ and doesnt automatically update as you move around. If that happens right click on the ‘Clusters’ feature in ‘Places’ and select Refresh, doing that once seems to fix it!

Thanks to Geocubes to the amazing service that powers this – highly recommended if you have larger numbers of features to display!

MGRS coordinate entry

Friday, December 12th, 2008

Prompted by one of those infernal* posts on a Google Maps group, have finally been inspired to cobble together a MGRS -> Lat/Long conversion. It seems trvial, the UTM->MGRS conversion is quite simple, but there are a few gotchas!, so after much puzzleing over some C code from GeoTrans (and Trial&Error for good measure!), I think** have something working.

Anyway try it out here: 

Coordinate Flyto for Google Earth/Maps

also added today is a tickbox to open the result direct in Google Maps (rather than Google Earth). 

I’ll tidy it up and post the code, which is based on the PHPcoord conversion class, incase others have a use. I’ll also add it to the API – and possibly even back port it into Perl for use on the on site coordinate covertor. 

Finally possibly of interest is the realisation that the MGRS Google Earth Gridlines layer, will actully load directly in Google Maps, so:

MGRS Gridlines in Google Maps

the lines should update a few seconds after dragging/zooming. Ideally will suppress the big blue pins (which are show as numbers in GE) , but not tonight…



* in the nicest possible sence!

** I tested it by converting utm->mgrs->utm globally on a worldwide 1 degree grid – all check out with absolute precision.

Google Earth: Flight Simulator GPS Arrow

Saturday, July 26th, 2008

Behold a new network link from

Google Earth GPS

Once you have chosen a destination point, a small arrow will be shown in the bottom right of your Google Earth window, which points towards your destination! Ideal for use in the flight simulator, where you can play follow the arrow.

Note it only updates every 5 seconds, and comes with all sorts of disclaimers such as don’t use in life or death situations.

Where in the world are people looking? part 2

Saturday, March 1st, 2008

Following part 1 yesterday, have now plotted some maps per individual service.

Flash Earth Layer


  1. Google Earth does request a link for the initial view on startup – that point in the Atlantic.
  2. I guess North American installs have two slightly different default placemarks; has it changed, or maybe Canada gets a different one?
  3. But that doesnt explain the lines from there – people must have their links set to periodically refresh?
  4. Germany likes FlashEarth (or rather people like looking at Germany with FlashEarth link enabled)
  5. Many people leave the links open even when not in use – e.g. the British Isles links show worldwide usage, notably over Russia.
  6. MGRS notably has high usage in two areas.
  7. The Far east likes to know what direction they facing.

Update: In a total forehead slapping momont realised these unprojected whole world images, are exactly what is used in GroundOverlays, so the the pages now include links to view the higher resolution images directly in Google Earth, duh!

Example: View In Google Earth (don’t forget to try adjusting the tranparency slider!) [Update, to fix broken link, sorry!]

Geohash converter; API limit upped

Thursday, February 28th, 2008

Recently was released; the premise: short easy links that encode a location. The website offers instant conversion, but as the algorithm is Public Domain, Lordelph was quick to implement a php class, and I could quickly add the conversion to my existing conversion API. Convert a postcode to geohash anyone?

Updating the documentation noticed the API was last updated in 2005 – gulp, how time flies! Anyway as a celebration have upped the throttling on the API, now three times the previous limit. Running on much improved hardware than when it was launched so maybe the limit can be tweaked even more, will run some analysis of its impact – it already seems the API is a minor part of the overall load on the server – particularly against the Google Earth layers. : short (network) links for KML

Wednesday, February 27th, 2008

For a project wanted to create a small script to make generation of network link wrappers to KML content easy, but then I thought could have it create ‘short’ urls, ala, to make reuse easy. But then thought if going to that trouble might as well make it a full blown mini application, so I give you :: Short Links to KML content

I think it turned out pretty well (esp for just an evenings hacking) – it even has an element of ‘design’ – which is saying something! : short (network) links for KML

So if you want to create a network link wrapper with the minimum of fuss, and might enjoy having it as a short url, then give it a go.

Of course there are still a few things that could be done, but will leave these for another day…

  • Auto discover the ‘title’
  • Intelligence for ‘My Map’s – eg support multiple pages, and maybe some other KML types.
  • Actually make the admin page useful – display stats (they are stored already) , and edit the link
  • A public gallery of popular/new etc links – maybe even some sort of search – esp if we can do rudimentary indexing of the content (when fetch the title)
  • Support for named links eg could choose
  • Complete the support for the various network link options – particulaly making it easier to select the appropriate settings.
  • Maybe even support for forwarding the link by email (and/or sending the admin link to yourself)
  • Any thing you can think of :) – let me know!

Playing with (geo-enabled) Full-Text Searches

Thursday, December 20th, 2007

Recently I have been playing a lot with Sphinx full-text search engine, in particular with regard to indexing the Geograph archive. (a bit of background – Geograph has a fairly good homegrown site text search – but its not full text, so many queries will not return that many results – not to mention been based on MySQL ‘like’, so is pretty slow – so a full text search is the next level). And I have to say I am liking it a LOT, in fact I would say I am a fanboy :)

So to that end of created a whole bunch of demos based around the flexible indexing it provides, location based searching is even possible!

At the most basic is simple text based search, one point of note, there is no pagination, simply add more keywords (including negative) or grid references to refine the selection.

Next is a ‘auto-complete’ style image finder, this is designed to find ‘that image’ quickly, in a similar way to the above but shows the results in a autocomplete box immediately!

A refinement of the first is search with location, this allows you limit the search to near a particular Grid References – this is particully cool in that there is Sphinx powered auto-complete for place names for finding GRs. (a real auto-complete not a like the search in the previous one pretending to be one)

This is all building towards the Illustrator demo. Which from a block of text attempts to find relevent images. The idea is that a (geolocated) news article, walking route, place description and such could be automatically have relevent(ish) images shown. (an example demo here)

(a few more ‘toys’ can be found in GeographTools!)…. Try them out and let me know how you get on…

I have learnt a lot about search indexing from this, including how to perform location searches in the index (I know latest versions of sphinx include a lat/long based geosearch – but I think this r-tree method in text has better scalability), and how to create an autocomplete function with sphinx. If anybody is interested in these, they will eventually make it into the geograph codebase, or let me know and I might make a separate post.

Interestingly (huh?), it was actually creating a ‘autocomplete’ textbox for finding trigpoints (which included the forerunner to the sphinx location search in but implemented in mysql), is actually what inspired me to actually go the trouble if figuring out how to install Sphinx on linux, which I have been interested in for a long time! – that is also now sphinx powered for text searches :)

As a side note have now reached the ‘linux sysadmin’ level that I can compile it on Geographes servers, yay! But I do worry for the sanity of others due to this (a little knowledge is a dangerous thing!)

MGRS fix for < 1km gratitules

Thursday, November 29th, 2007

Following the update in July for hiding the 100km numbers from the gridline labels, it seems the same fix broke the sub KM lines, thanks to a comment on the original post for letting me know.

Anyway, the online hosted version has been updated (and checked at all scales as best I can!), and the same file pushed into the downloadable zip for the offline version, so can download the latest code from the file. And just for completeness the diff.

FlashEarth+GoogleEarth Part 2

Wednesday, August 29th, 2007

Following the amazing interest in the networklink for embedding FlashEarth inside Google Earth 4.2, have performed a few updates…

  1. The zoom is a LOT closer matched, this is thanks to the inspiring formula: $z = (-1.4956 * log($range/1000) + 16.529; (yes really!); brought to you by the wonder of plotting a graph in Excel, creating a trendline, turning on the formula display option. Probably could get an even better match with more patience, but probably about as good as it going to get really.
  2. Removed the title and the directions links, makes the balloon looks a bit nicer, couldn’t get it to go transparent for me tho… suggestions welcome.
  3. When the network link is now a folder so can choose the size of your popup :) The previous version is now the ‘medium’ one.

Though the magic of the web, you get 1) and 2) automatically, if want 3) just download it again:

Open in Google Earth (Required: GE 4.2 Windows)

Thanks again to everyone involved.

This month I have mostly been scaling

Wednesday, August 15th, 2007

… a website for more traffic that is. This is something a little off-topic perhaps for this blog, but it might be of interest to a few so will document a few tricks have learnt, in tweaking Geograph to cope with more traffic as it’s daily visitors and hits continue to climb. If you are not familiar with Geograph, or not a System-Admin (or budding – like me!), then probably can stop reading now!

First a little background, Geographs code started very humble, and coded to work off a single server, later with OS sponsorship we upgraded to multiple servers to cope with increasing traffic. This was done with a single larger server for Database and photo storage, and then multiple commodity webservers (with a front end load-balancer) More. This worked well for a number of months, but simply the DB/NAS server couldn’t cope with the increasing DB load, and bandwidth for serving all hundred of thousands of photos.

  • Split the database, a small quick win is/was split off php sessions and gazetteer queries to a second Database. Sessions of course have lots of writes, so where tending to saturate the main db, this perhaps reduced its load by a 3rd!
  • Cache Images on the separate servers. The servers aren’t big enough to house a copy of the full archive, but thumbnails are certainly more manageable. Seeing as thumbnails actually account for about 60-70% of the raw hits to the site, this is a potential win, as previously each server would have to seperatelly fetch individual images off the NAS. We use Apache as the webserver, so could easily create another simple VirtualHost to serve thumbnails, a empty DocumentRoot save images, with a simple 404 handler to fetch and store images not ready copied. This greatly reduces load on the NAS as its not having 3x servers fetching random thumbnails. (this also paves the way to move away from full-blown Apache simply for static content)
  • Cache stuff in Memory – with Memcache. Related to the above point, quite a bit of load is actually random disk IO to determine image sizes as this requires reading the jpeg data. Caching this all in memory is good. Memcache can easily distribute it cache across multiple machines, so even losing a server means only part of the cache needs rebuilding. We also use ADODB as a database abstraction layer, with the latest version it has support to use Memcache for its caching, great! Last up is to do lots of application level caching on key places. Of course sessions and also the templating system (smarty) could benefit from memcache, but one step at a time!
  • Optimise the HTTP headers. There are lots of tweaks and stuff here that can be done to lower the bandwidth and improve external cacheability of objects. This post is getting quite long so I think that might be a separate post…
  • Optimize the slow queries. And last of not least, learn to love going though the log of slow database queries, and really stepping on the slow ones. This of course is an ongoing project. I found a script to summarize these, but it seems that mysql 5 at least comes with its own equally good one!

This list isn’t exhaustive, and of course is an on going project, always more can be done…