nearby.org.uk blog

Recently I have been playing a lot with Sphinx full-text search engine, in particular with regard to indexing the Geograph archive. (a bit of background - Geograph has a fairly good homegrown site text search - but its not full text, so many queries will not return that many results - not to mention been based on MySQL ‘like’, so is pretty slow - so a full text search is the next level). And I have to say I am liking it a LOT, in fact I would say I am a fanboy

So to that end of created a whole bunch of demos based around the flexible indexing it provides, location based searching is even possible!

At the most basic is simple text based search, one point of note, there is no pagination, simply add more keywords (including negative) or grid references to refine the selection.

Next is a ‘auto-complete’ style image finder, this is designed to find ‘that image’ quickly, in a similar way to the above but shows the results in a autocomplete box immediately!

A refinement of the first is search with location, this allows you limit the search to near a particular Grid References - this is particully cool in that there is Sphinx powered auto-complete for place names for finding GRs. (a real auto-complete not a like the search in the previous one pretending to be one)

This is all building towards the Illustrator demo. Which from a block of text attempts to find relevent images. The idea is that a (geolocated) news article, walking route, place description and such could be automatically have relevent(ish) images shown. (an example demo here)

(a few more ‘toys’ can be found in GeographTools!)…. Try them out and let me know how you get on…

I have learnt a lot about search indexing from this, including how to perform location searches in the index (I know latest versions of sphinx include a lat/long based geosearch - but I think this r-tree method in text has better scalability), and how to create an autocomplete function with sphinx. If anybody is interested in these, they will eventually make it into the geograph codebase, or let me know and I might make a separate post.

Interestingly (huh?), it was actually creating a ‘autocomplete’ textbox for finding trigpoints (which included the forerunner to the sphinx location search in but implemented in mysql), is actually what inspired me to actually go the trouble if figuring out how to install Sphinx on linux, which I have been interested in for a long time! - that is also now sphinx powered for text searches

As a side note have now reached the ‘linux sysadmin’ level that I can compile it on Geographes servers, yay! But I do worry for the sanity of others due to this (a little knowledge is a dangerous thing!)

 Only a quick post as its late, but just noticed this report had finished generating, took rather a long time, as just found our stats package has been mangling GE’s useragent string - anyway enough rambling, I know you just want to view the stats:

 Dec 2007 :: Google Earth versions accessing links on nearby

… make of that what you will - haven’t really drawn any conclusions from this…

(last set)

Have finally got round to (almost) sorting a issue that has plaguing this site for a while, namely a lack of header/footer for site pages. Primarily from laziness, but also due to the mixmash of server scripting languages used to power Nearby. Anyway the upshot is can now relatively easily edit the top/bottom menu areas. So to celebrate there is now far more links at the top of the page.

Have also taken the opportunity to publish a revised homepage I tried creating a while ago, I am not totally happy with it but think it possibly an improvement (or at least a declutter) of the old one, sadly no new features released tho!
(yes this is the first time in over a year an a half have done substantial development in areas other than the Google Dabbles or GeographTools pages)

Following the update in July for hiding the 100km numbers from the gridline labels, it seems the same fix broke the sub KM lines, thanks to a comment on the original post for letting me know.

Anyway, the online hosted version has been updated (and checked at all scales as best I can!), and the same file pushed into the downloadable zip for the offline version, so can download the latest code from the file. And just for completeness the diff.

Owing to the Flash Earth layer thingy published a few days ago, you might notice things a little choppy around here at the moment… my own little corner of the web is not normally this busy. Currently have about 3-4 times the normal traffic levels, which is about 200 visitors at any one time. Just the layer has been downloaded by about 1,200 people and generated about 70,000 hits. So far beating the Cloud and sky layer from a while ago by about x2.

Fortunately I been working on scaling rather a lot lately (on Geograph and a bit at work), so might have to put some things into practice ‘ere.

What if the current view in Google Earth was quickly viewable in other maps, such as those made accessible by FlashEarth? Well, now it is. As you move around the globe a little white arrow follows you around, simple click it to get an approximation of the current view in FlashEarth in a popup balloon.

Open in Google Earth

Total credit to Valery35, for the concept (including a screenshot) on the Google Earth Community. (and to all the people made all the bits that could be pieced together in this 15min hack)

Yesterday Geograph reached a significant milestone, getting photographs for a whole myriad. That’s a whole 100×100km square, or 10,000 squares.

Ok there have been a few smaller myriads complete for a while, but SP is a fully landlocked square so represents a significant achievement.

To celebrate: here is a zoomable flash wotsit to showcase all those glorious squares…

Recently I have been playing with YSlow quite a bit, mainly for Geograph. A useful feature is it gives the load time for every page (without doing a full scan) in the status bar and I happened to visit this blog and the results where quite surprising.

Geopress which I have installed for adding GeoRSS to rss feeds and creating kml files, (and geotagging the post itself, which is custom), actually includes it’s javascript on each an every page! So 6kb for the core, 34kb for mapstraction, and then because I had blindly entered a API key for Google and Yahoo, get both of their apis thrown in (about 150kb I think), and no maps for all that effort!
I use a Google map for geocoding the post in the first place, so don’t want to blindly disable the whole lot, so as a quick bodge, I just commented out the following line in geopress.php

add_action(’wp_head’, array(’GeoPress’, ‘wp_head’));

leaving the ‘admin_head’, so that it would get included in the admin , which is just me.

If I knew more I would digg in and make it a preference, or better yet just make it auto-detected. But posting this here in the hope it useful to someone.

… I’m a lazy blogger, I don’t bother posting things I find, as I naively think people everybody will have already read it.

I’ve also just started using Google Reader in a bigger way, and as such as a small experiment will start sharing items I think are relevent to this blog, get the feed here:

http://feeds.feedburner.com/NearbyIsReading

(its via feedburner to make a nicer url, as well as to get some stats to see if it worth me doing this…)

Ever since releasing the MGRS layer for Google Earth, I’ve been asked a number of times (well about 10) about tweaking the numbers on the gridlines, to be more ‘MGRS like’. To be honest havent had a clue what this means, but a recent email made me twig whats going on.

Basically the numbers included the hundreds of KM, but in fact that number is already represented in the Grid Letters so doesnt make sence to include again (it was there because the layer was an almost direct port of the UTM code, which of course needs it).

So the server hosted version has been updated, and also the file inside the zip for the offline version (which are in fact identical code!), and if really interested heres the diff.

Enjoy!