Archive for the ‘Code’ Category

Geograph Javascript Playgrounds

Friday, July 1st, 2011


I often have little ideas for new ways to display images from Geograph, but its a pain building prototypes.

It usually involves finding some other part developed prototype or feature, that I can use as a template. Then have start up a code editor, and get connected to a server. Think of a name to call the file. Strip out the bits not needed for the new application. Then have to remind myself the APIs available. Work out how to call the APIs etc, etc, etc.

… well now I’ve began work on a platform to make this easier. Online code editor – quick forking of current projects. Easy wrapped access to relevant APIs, jQuery already loaded.

No more worrying about what to call the file, or making sure I keep backups of previous versions (should want to find a previous one!). This is all handled automatically. Setup to handle browser caching and invalidation on new versions. No worrying about stale code.

It’s primarily for my own use, but thought may as well open it up so others can use it build quick prototypes – or even fully fledged applications (based on Geograph images of course!).

The applications created all run client-side in browser – so utilizes the power of jQuery to create immersive and compelling interfaces.

Probably the best place to start is from a few of the early demos I built with the system:

Public Listed Applications

Please use them as starting points – or simply inspiration to build your own application! (They are not meant as complete applications!)

No special tools required – other than a browser. Firefox with Firebug, or Chrome (press F12!) recommended – to help catch javascript errors.

(note: it only wraps one Geograph API currently – loading all images in a gridsquare – more to be added)

Full UK postcodes in the API!

Friday, April 30th, 2010

Thanks to OS Opendata, the Nearby Converter API now can convert to and from full Unit UK postcodes!

So as not to break applications expecting Sector postcodes in the API, there is a new API endpoint:

http://api1.nearby.org.uk/api/convert.php

API requests against this domain will give the position of the full postcode, and also reverse geocode (return you the nearest) full postcode!

- The Original URL for the API is still available – and will continue to be. I’ve also taken the opportunity to add a version number to the URL. It’s been 4 years since I created this API, and learnt lots in that time, so should be able to create a better version 2 :) When introduce that don’t want to break compatibility.

Of course the ‘Coordinate Convertor‘ is now using CodePoint Open too – the end of having to pay a commercial provider!

(sorry this has taken so long, OS Opendata was introduced at the beginning of the month!, its been a busy month)

Will take the opportunity to duplicate the copyright message:

Contains Royal Mail data © Royal mail copyright and database right 2010.

As it serves as a useful thank you, for making this data available.

Geocube Geograph Clusters in Google Earth – part 2

Wednesday, July 15th, 2009

Now ready with the next version of the Geograph Google Earth Clusters layer mentioned yesterday.

Google Earth Layer Geograph Google Earth Clusters

It’s the same download link, if you already have a copy, just right click and select Refresh on the “Geograph Google Earth Clusters” feature.

This has a number of advancements:

  • Filterable – click the main title in the Places panel to open configuration options in the popup balloon. Includes options to filter by keyword or Geograph contributor. (Requires Google Earth version 5)
  • Cubes/Cloud – Geocubes offers two clustering options, can how choose between then on the same popup as above
  • Thumbnails – Now shows a image thumbnail in the placemark balloons.
  • Shows a few coarse clusters when not zoomed on the British Isles
  • Source code available! Download the GPL licenced PHP source code here

But its not all rosey – this version seems to suffer from an annoying bug, when first loads sometimes the layer ‘freezes’ and doesnt automatically update as you move around. If that happens right click on the ‘Clusters’ feature in ‘Places’ and select Refresh, doing that once seems to fix it!

Thanks to Geocubes to the amazing service that powers this – highly recommended if you have larger numbers of features to display!

ITM <-> WGS84 (<-> Irish Grid <-> British National Grid) PHP code

Wednesday, May 13th, 2009

With Ireland soon to be moving to ITM (Irish Transverse Mercator) coordinates, over IG (Irish Grid) that has been used since about 1965 in Ireland, many websites – including Geograph.ie and nearby.org.uk will be looking to support ITM coordinates – even if they still use Irish Grid.

So to that end have been looking for PHP code to do the conversion, but thanks to a little bit of inspiration from FieldenMaps.info’s Javascript Ireland Coordinate Converter (an excellent resource in its own right!), realised it would be simple to add to the conversion class we already use. In fact as ITM is based on GRS80 which is effectivly WGS84 the conversion was just a matter of plugging in the constants used by ITM, and the code just worked.

Better yet as Geograph is Open Source (and we got permission to open source the code based on OSGB’s conversion spreadsheet), the code is available for anyone to use as a PHP class. It should be trivial to add support to our Javascript class – which will do shortly!

Grab the class from Geograph SVN (or even better do a SVN checkout!) and put it somewhere safe. (but you will probably want to remove the “extends Conversions” bit from the definition – we use that for convience in Geograph source)

Then can just use it

$conv = new ConversionsLatLong;
list($lat,$lng) = $conv->itm_to_wgs84($e,$n);

But as we already use the same code for Irish Grid, and OSGB36 British National Grid, can also use it for them, and even IG< ->ITM conversion

list($lat,$lng) = $conv->itm_to_wgs84($e,$n);
list($lat,$lng) = $conv->irish_to_wgs84($e,$n,true);
list($lat,$lng) = $conv->osgb_to_wgs84($e,$n);

list($e,$n) = $conv->wgs84_to_itm($lat,$lng);
list($e,$n) = $conv->wgs84_to_irish($lat,$lng,true);

list($e,$n) = $conv->wgs84_to_osgb36($lat,$lng);
//even a short convience function to create a 10 figure grid reference
$gr10 = $conv->osgb36_to_gridref($e,$n);

See it used in anger here and here.

MGRS coordinate entry

Friday, December 12th, 2008

Prompted by one of those infernal* posts on a Google Maps group, have finally been inspired to cobble together a MGRS -> Lat/Long conversion. It seems trvial, the UTM->MGRS conversion is quite simple, but there are a few gotchas!, so after much puzzleing over some C code from GeoTrans (and Trial&Error for good measure!), I think** have something working.

Anyway try it out here: 

Coordinate Flyto for Google Earth/Maps

also added today is a tickbox to open the result direct in Google Maps (rather than Google Earth). 

I’ll tidy it up and post the code, which is based on the PHPcoord conversion class, incase others have a use. I’ll also add it to the API – and possibly even back port it into Perl for use on the on site coordinate covertor. 

Finally possibly of interest is the realisation that the MGRS Google Earth Gridlines layer, will actully load directly in Google Maps, so:

MGRS Gridlines in Google Maps

the lines should update a few seconds after dragging/zooming. Ideally will suppress the big blue pins (which are show as numbers in GE) , but not tonight…

 

 

* in the nicest possible sence!

** I tested it by converting utm->mgrs->utm globally on a worldwide 1 degree grid – all check out with absolute precision.

Geopress and no maps?

Saturday, August 25th, 2007

Recently I have been playing with YSlow quite a bit, mainly for Geograph. A useful feature is it gives the load time for every page (without doing a full scan) in the status bar and I happened to visit this blog and the results where quite surprising.

Geopress which I have installed for adding GeoRSS to rss feeds and creating kml files, (and geotagging the post itself, which is custom), actually includes it’s javascript on each an every page! So 6kb for the core, 34kb for mapstraction, and then because I had blindly entered a API key for Google and Yahoo, get both of their apis thrown in (about 150kb I think), and no maps for all that effort!
I use a Google map for geocoding the post in the first place, so don’t want to blindly disable the whole lot, so as a quick bodge, I just commented out the following line in geopress.php

add_action(‘wp_head’, array(‘GeoPress’, ‘wp_head’));

leaving the ‘admin_head’, so that it would get included in the admin , which is just me.

If I knew more I would digg in and make it a preference, or better yet just make it auto-detected. But posting this here in the hope it useful to someone.

This month I have mostly been scaling

Wednesday, August 15th, 2007

… a website for more traffic that is. This is something a little off-topic perhaps for this blog, but it might be of interest to a few so will document a few tricks have learnt, in tweaking Geograph to cope with more traffic as it’s daily visitors and hits continue to climb. If you are not familiar with Geograph, or not a System-Admin (or budding – like me!), then probably can stop reading now!

First a little background, Geographs code started very humble, and coded to work off a single server, later with OS sponsorship we upgraded to multiple servers to cope with increasing traffic. This was done with a single larger server for Database and photo storage, and then multiple commodity webservers (with a front end load-balancer) More. This worked well for a number of months, but simply the DB/NAS server couldn’t cope with the increasing DB load, and bandwidth for serving all hundred of thousands of photos.

  • Split the database, a small quick win is/was split off php sessions and gazetteer queries to a second Database. Sessions of course have lots of writes, so where tending to saturate the main db, this perhaps reduced its load by a 3rd!
  • Cache Images on the separate servers. The servers aren’t big enough to house a copy of the full archive, but thumbnails are certainly more manageable. Seeing as thumbnails actually account for about 60-70% of the raw hits to the site, this is a potential win, as previously each server would have to seperatelly fetch individual images off the NAS. We use Apache as the webserver, so could easily create another simple VirtualHost to serve thumbnails, a empty DocumentRoot save images, with a simple 404 handler to fetch and store images not ready copied. This greatly reduces load on the NAS as its not having 3x servers fetching random thumbnails. (this also paves the way to move away from full-blown Apache simply for static content)
  • Cache stuff in Memory – with Memcache. Related to the above point, quite a bit of load is actually random disk IO to determine image sizes as this requires reading the jpeg data. Caching this all in memory is good. Memcache can easily distribute it cache across multiple machines, so even losing a server means only part of the cache needs rebuilding. We also use ADODB as a database abstraction layer, with the latest version it has support to use Memcache for its caching, great! Last up is to do lots of application level caching on key places. Of course sessions and also the templating system (smarty) could benefit from memcache, but one step at a time!
  • Optimise the HTTP headers. There are lots of tweaks and stuff here that can be done to lower the bandwidth and improve external cacheability of objects. This post is getting quite long so I think that might be a separate post…
  • Optimize the slow queries. And last of not least, learn to love going though the log of slow database queries, and really stepping on the slow ones. This of course is an ongoing project. I found a script to summarize these, but it seems that mysql 5 at least comes with its own equally good one!

This list isn’t exhaustive, and of course is an on going project, always more can be done…

Tweaking the number of the MGRS layer

Monday, June 25th, 2007

Ever since releasing the MGRS layer for Google Earth, I’ve been asked a number of times (well about 10) about tweaking the numbers on the gridlines, to be more ‘MGRS like’. To be honest havent had a clue what this means, but a recent email made me twig whats going on.

Basically the numbers included the hundreds of KM, but in fact that number is already represented in the Grid Letters so doesnt make sence to include again (it was there because the layer was an almost direct port of the UTM code, which of course needs it).

So the server hosted version has been updated, and also the file inside the zip for the offline version (which are in fact identical code!), and if really interested heres the diff.

Enjoy!

OpenSearch (with Geo) Description File Tester

Thursday, June 21st, 2007

(Very) Simplistic tool for trying out a OpenSearch Description file :

http://www.nearby.org.uk/opensearch/

… includes a Gmap to make defining the lat/long and/or bbox easy. (if the file has geo extensions)

(Just submit the same file again, if you update it. btw don’t submit a definition file you don’t want to become public, but remember it is called ‘open’ ;)

I realised after mentioning we updated the description file on www.geograph.org.uk, that it wasn’t fully tested, (hate testing, but not adverse to spending 2 hours scripting a tester), and in fact this tool highlighted a mistake!

Hope it of interest, feedback welcome,

3DConnexion on 2D Google Maps…

Thursday, March 22nd, 2007

Thanks to Frank for letting us know about the new SDK for the 3DConnexion devices, including the SpaceNavigator, I’ve been waiting for this for a while as wanted to try it for websites like Google Maps. Even nicer is the SDK includes a Javascript demo – so makes developing for Google Maps Easy!

Enough already, I want to see the Demo!

WARNING: Internet Explorer – and you might have to enable ActiveX controls to run. Do so at your own risk! I won’t intentionly do anything bad, but can’t promise that 3DConnexion won’t (but doubtful they would either) – Recommend you add Nearby to your trusted sites and then allow controls to run in the trusted zone.

Or just download the file to your local drive. (right click the above link and select Save As) and run it from a local file – didn’t have to jump thought the hoops to get it to run there!

Todo: The zooming isn’t as slick as it could be (possibly) and would be nicer to have more control of the speed. Also the silly (depending on your point of view!) warnings from IE7, which I wonder if there is a better way of loading the objects to make it more compatible.