Archive for August, 2007

Int—er—mitt—ent — Ser—vi—ce….

Thursday, August 30th, 2007

Owing to the Flash Earth layer thingy published a few days ago, you might notice things a little choppy around here at the moment… my own little corner of the web is not normally this busy. Currently have about 3-4 times the normal traffic levels, which is about 200 visitors at any one time. Just the layer has been downloaded by about 1,200 people and generated about 70,000 hits. So far beating the Cloud and sky layer from a while ago by about x2.

Fortunately I been working on scaling rather a lot lately (on Geograph and a bit at work), so might have to put some things into practice ‘ere.

FlashEarth+GoogleEarth Part 2

Wednesday, August 29th, 2007

Following the amazing interest in the networklink for embedding FlashEarth inside Google Earth 4.2, have performed a few updates…

  1. The zoom is a LOT closer matched, this is thanks to the inspiring formula: $z = (-1.4956 * log($range/1000) + 16.529; (yes really!); brought to you by the wonder of plotting a graph in Excel, creating a trendline, turning on the formula display option. Probably could get an even better match with more patience, but probably about as good as it going to get really.
  2. Removed the title and the directions links, makes the balloon looks a bit nicer, couldn’t get it to go transparent for me tho… suggestions welcome.
  3. When the network link is now a folder so can choose the size of your popup :) The previous version is now the ‘medium’ one.

Though the magic of the web, you get 1) and 2) automatically, if want 3) just download it again:

Open in Google Earth (Required: GE 4.2 Windows)

Thanks again to everyone involved.

Google Earth 4.2 + FlashEarth equals…

Tuesday, August 28th, 2007

What if the current view in Google Earth was quickly viewable in other maps, such as those made accessible by FlashEarth? Well, now it is. As you move around the globe a little white arrow follows you around, simple click it to get an approximation of the current view in FlashEarth in a popup balloon.

Open in Google Earth

Total credit to Valery35, for the concept (including a screenshot) on the Google Earth Community. (and to all the people made all the bits that could be pieced together in this 15min hack)

Geograph completes its first whole Myriad

Monday, August 27th, 2007

SP Myriad - Geograph Coverage Aug 2007 - (c) Geograph Creative Commons LicencedYesterday Geograph reached a significant milestone, getting photographs for a whole myriad. That’s a whole 100x100km square, or 10,000 squares.

Ok there have been a few smaller myriads complete for a while, but SP is a fully landlocked square so represents a significant achievement.

To celebrate: here is a zoomable flash wotsit to showcase all those glorious squares

Geopress and no maps?

Saturday, August 25th, 2007

Recently I have been playing with YSlow quite a bit, mainly for Geograph. A useful feature is it gives the load time for every page (without doing a full scan) in the status bar and I happened to visit this blog and the results where quite surprising.

Geopress which I have installed for adding GeoRSS to rss feeds and creating kml files, (and geotagging the post itself, which is custom), actually includes it’s javascript on each an every page! So 6kb for the core, 34kb for mapstraction, and then because I had blindly entered a API key for Google and Yahoo, get both of their apis thrown in (about 150kb I think), and no maps for all that effort!
I use a Google map for geocoding the post in the first place, so don’t want to blindly disable the whole lot, so as a quick bodge, I just commented out the following line in geopress.php

add_action(‘wp_head’, array(‘GeoPress’, ‘wp_head’));

leaving the ‘admin_head’, so that it would get included in the admin , which is just me.

If I knew more I would digg in and make it a preference, or better yet just make it auto-detected. But posting this here in the hope it useful to someone.

New Earth

Wednesday, August 22nd, 2007

A next version of Google Earth is out! This in the one that implements the things we got a sneak preview of with KML2.2. Unfortunatly it would have to be today, the one day I cant actully play with it, so will have to hold off any try it out propelly another day :(

I’ll forward you over to Ogle Earth to get the lowdown on the new features.

I just hope that the 3DConnexion SpaceNavigator not working in Photo (or Sky!) mode is just a glitch…

Building the UK one block at a time!

Monday, August 20th, 2007

Ever played Tetris? Know your geographical areas? If so then its time to play Statetris,

Statetris is an interesting game mixing aspects of the popular game ‘Tetris’ and geography. Instead of positioning the typical Tetris blocks, you position states/countries at their proper location.”

…even better a UK version has just been released!

As the project diversifies to new areas, the creators are facing some interesting challenges to cope with our diverse world. For example the UK is a problem as its so tall, so doesn’t fit on screen as nice as some other areas, not to mention our confusing and changing geography! The Netherlands on the other hand is quite small, so has capitals added to add interest.

What’s your best time?

“The Wrekin” Panorama

Sunday, August 19th, 2007

As a little experiment in super high resolution panoramas, created a 61215x3105px image, stitched together from a number of 10Mpix images. I’ve uploaded here as a Zoomify Viewer so make viewing it quick and easy.

I’ll try and create a PhotoSpot out of it for Google Earth when get a chance, but would love to try this with the Image Pyramid and Photo Overlay system that might be coming soon!

Update: Uploaded the image to GigaPan.com, which takes the heavy lifting of getting the pano into Google Earth 4.2 plus. (which can display it using the Pyramid system mentioned!) Highly recommended. It’s possibly a little buggy, but the new version of GE has only been out a day!

This month I have mostly been scaling

Wednesday, August 15th, 2007

… a website for more traffic that is. This is something a little off-topic perhaps for this blog, but it might be of interest to a few so will document a few tricks have learnt, in tweaking Geograph to cope with more traffic as it’s daily visitors and hits continue to climb. If you are not familiar with Geograph, or not a System-Admin (or budding – like me!), then probably can stop reading now!

First a little background, Geographs code started very humble, and coded to work off a single server, later with OS sponsorship we upgraded to multiple servers to cope with increasing traffic. This was done with a single larger server for Database and photo storage, and then multiple commodity webservers (with a front end load-balancer) More. This worked well for a number of months, but simply the DB/NAS server couldn’t cope with the increasing DB load, and bandwidth for serving all hundred of thousands of photos.

  • Split the database, a small quick win is/was split off php sessions and gazetteer queries to a second Database. Sessions of course have lots of writes, so where tending to saturate the main db, this perhaps reduced its load by a 3rd!
  • Cache Images on the separate servers. The servers aren’t big enough to house a copy of the full archive, but thumbnails are certainly more manageable. Seeing as thumbnails actually account for about 60-70% of the raw hits to the site, this is a potential win, as previously each server would have to seperatelly fetch individual images off the NAS. We use Apache as the webserver, so could easily create another simple VirtualHost to serve thumbnails, a empty DocumentRoot save images, with a simple 404 handler to fetch and store images not ready copied. This greatly reduces load on the NAS as its not having 3x servers fetching random thumbnails. (this also paves the way to move away from full-blown Apache simply for static content)
  • Cache stuff in Memory – with Memcache. Related to the above point, quite a bit of load is actually random disk IO to determine image sizes as this requires reading the jpeg data. Caching this all in memory is good. Memcache can easily distribute it cache across multiple machines, so even losing a server means only part of the cache needs rebuilding. We also use ADODB as a database abstraction layer, with the latest version it has support to use Memcache for its caching, great! Last up is to do lots of application level caching on key places. Of course sessions and also the templating system (smarty) could benefit from memcache, but one step at a time!
  • Optimise the HTTP headers. There are lots of tweaks and stuff here that can be done to lower the bandwidth and improve external cacheability of objects. This post is getting quite long so I think that might be a separate post…
  • Optimize the slow queries. And last of not least, learn to love going though the log of slow database queries, and really stepping on the slow ones. This of course is an ongoing project. I found a script to summarize these, but it seems that mysql 5 at least comes with its own equally good one!

This list isn’t exhaustive, and of course is an on going project, always more can be done…