thread dump

The previous post elicited a comment from Andy at CloudBerry Lab:

I am a developer of another online backup product powered by Amazon S3 http://cloudberrydrive.com/ that we are going to release to beta soon. I would appreciate if you take a look and may be post a review on your blog.

Well I did take a look, but the CloudBerry product isn’t really what I’m looking for.  If all you want/need is a Windows-only client for accessing your S3 buckets directly and copying/moving content, then this product might well be just right.  But it doesn’t really compete with DropBox or JungleDisk.  Andy: if your product were closer to DropBox, and cross-platform, then I would have reviewed it.  As it is, I can’t pass judgement.

If you have time to plough through a very long blog thread, check out Jeremy Zawodny’s excellent piece on Amazon S3 tools. The discussion thread contains links to most (maybe all) the tools out there.

I’ve also been reading the DropBox forums. It’s good to see plenty of other people have requested features I miss (e.g. ability to configure DB to ignore certain file patterns).  There is an upcoming features wiki page which lists some things coming ‘soon’.  Nothing on using your own S3 space though.

There are quite a few network-drive products out there competing for our attention these days.  I’ve tried quite a few (including Box.net and SkyDrive), but narrowed the choice down to the two which seem the best: JungleDisk and DropBox.

JungleDisk attracts me mostly because I can use my existing Amazon S3 storage behind the JungleDisk tool and pay only for the space I actually use. (Note, JD now also uses RackSpace CloudFiles, which actually looks even better than S3).  DropBox is free for 2GB, then a hefty 99 USD annually, for 50GB.  Round one to JungleDisk, in my view.

Then we come to the user-interface. Both tools integrate with Windows and Linux, JD using drive mapping to expose the storage and DropBox using a special folder inside My Documents, with icon overlays to indicate file status.  Both support drag/drop access and run a small tray-resident UI application.

But DropBox is just so, so much nicer to use than JD in the everyday Windows context.  It feels better integrated and the UI seems cleaner.  Other folk have blogged about this difference and I must concur – DropBox has the edge.

Now to the subject which prompted this post in the first place. Neither of these products appears to handle proxy servers particularly well, especially when switching between proxy / no-proxy.  If I restart Windows and forget to switch off the proxy in JD, here’s the mess I’m greeted with when Windows starts:

Yuk. Can’t it simply notice that the proxy isn’t responding, log the fact / decorate the tray icon, and leave it for me to sort out?  It gets worse: if I click on the links (for more information) look what I get:

Is this really what they want the user to see?  This is awful.

DropBox is slightly better, but still doesn’t work properly if I leave the proxy on, and restart. No nasty dialogs, but the network connection isn’t resolved, even if I set it to ‘auto-detect proxy settings’ which according to the DropBox site should use the IE settings. Why can’t these tools auto-detect proxies properly?

DropBox files are cached on the local machine which means if the network is down I can still work on all my files locally, and re-sync when I next connect.  JungleDisk does cache your files, but in a pretty inaccessible way in your profile.  The path will be something like C:\Documents and Settings\<user>\Application Data\JungleDisk\cache\e9998872111157539d8880eca4456345-default

Another good feature of DropBox which isn’t available in JungleDisk is sharing files and folders: in JD, everything is private.

DropBox gets so many things right. The one and only feature I want from JungleDisk is the S3 / CloudFiles backing store. Obviously, the DropBox business model is built around the 99 USD annual charge so I don’t know whether this can/will ever happen.

Bit disappointed with the latest VS Express 2008 installer – I’ve download the ‘kitchen sink’ ISO image (containing all the VS Express editions), but when I try to install VC#, I get these errors:

and this one:

Because this is a new laptop (therefore a fresh Vista install) I did wonder whether this was a Windows Installer service issue, but the service is there and appears to be the latest version.  I’m puzzled, and more than slightly irritated.  It must be something simple – I’ll no doubt kick myself when I discover what it is.  Meantime, I’ll try the network install.

I recently got a shiny new laptop, which is great, but usually this means installing and configuring a lot of stuff before it’s really a comfortable place to be.  I’m sure this is familiar to most folk.  I expected one of the biggest challenges to be Firefox, as I have a number of plugins, a lot of passwords and the usual bookmarks and preferences. 

Luckily, I found FEBE, a Firefox addin which effectively solves this problem.  The addon can be found on the Mozilla site here, but the author’s own site has the latest version – there seems to be quite a big version gap.

To take all of my settings to the new machine I used the ‘Full Profile’ option in FEBE which creates a single file containing absolutely everything: I did try experimenting with a subset but it wasn’t successful.  If you’re not familiar with Firefox profiles I strongly suggest reading this section of the FAQ on the author’s site and follow the instructions.

I use Google Docs and Google Reader all the time. With Google Gears, these are set to become even more useful because they’ll work when I’m offline. The Reader already works this way (see this post for more), but not Docs + Spreadsheets yet. Rumour has it that this is coming. If they solve the sync problem properly, this will be seriously good: Docs is fine for note-taking and certainly good enough for blog posting. And of course they must extend Gears to GMail.

Web 2.0, Ajax + Gears (+ whatever server-side stuff you use to generate all this) may be good enough technology for building browser-applications which work offline, but underneath the covers it’s still based on HTML, grinding out a ton of hard-to-debug JavaScript, and running inside the browser frame. For the poor developer, even though dedicated folk spend hours creating libraries like script.aculo.us (and Google’s own GWT ) the experience of creating these applications remains pretty dire. And the result of all this extraordinary effort is still something which doesn’t even approach the sophistication of the equivalent desktop application. As I said, I really like Google Docs, but it isn’t Word, and won’t ever be.

Then there’s Adobe Flash and ActionScript, which has been around for a long time, is well established and has been used to create quite complex browser-hosted applications, such as Gliffy, which is (a bit) like Visio in a browser frame. Flash is cross-platform, as well as cross-browser, but still there’s quite a gap between developing for Flash, and developing for the desktop. Now Adobe Labs is promoting AIR (the technology formerly known as Apollo). This looks like a warmed-over Flash, and appears to be based still on JS and HTML.

When I first heard about Silverlight, I wondered why Microsoft appeared to be tilting at the Flash/ActionScript windmill? Why would they expect developers of browser-hosted, graphical presentations to switch away from Flash? But that’s not really what Silverlight is about: this is about creating a platform for Rich Internet Applications (RIAs) based on the .NET platform, which makes this potentially much more interesting, especially if development can be done in C# using Visual Studio, rather than mashing together markup and script. The runtime appears to be a a slightly reduced version of the CLR, supporting WPF/XAML as well as the traditional HTML/JS mixture. Quite remarkable, in a 1.1MB download. Rather than repeat more technical details here, I’ll refer you to the Silverlight architecture summary paper.

Silverlight already runs on Mac as well as Windows, and in all the popular browsers. Could it be ported to Linux/Unix? The Mono project has proved the portability of the runtime, so it doesn’t sound like such an outrageous suggestion.

Upgrading successfully from 2.0.2 to 2.2 was not as straightforward as it should have been.

I followed all the instructions, disabling all the plugins and setting the theme back to the original default, but when I ran the wp-admin/upgrade.php script I got a lot of database errors.  The blog worked, but the categories were missing and the admin pages which refer to them showed errors.

These seem to be due to database permissions issues.  Googling for a solution I found a reference or two (e.g. this one), so I restored the old DB (what a good thing I did the backup!) and used phpMyAdmin to change the db-user permissions to include ALTER.  This appeared to fix the problem because the upgrade script worked perfectly.

My main gripe is that despite the (tedious, manual) DB fix, most of my posts appeared to lose their category tags.  Eventually, these appeared again, but I’m not sure exactly why.

The upgrading process is irritating. WordPress is such a fine piece of work overall, it seems such a shame that upgrading should have been be so clumsy and buggy.

The Google technologies just keep on improving: I’ve just started to use the new version of Google’s feed reader, and I really think this is the browser-based reader which makes fat-client reader applications redundant (well, for me at least).  I have been using RSS Owl, an Eclipse RCP application which I think is very good (and cross-platform, and free), but I really want to store my RSS feed subscriptions on the web rather than copy a file around, and my requirements for a reader are pretty basic.

How much further will I want to go with this?  I already use GMail and the Google Calendar, which effectively makes Outlook redundant, and the Google Notebook for capturing material and scribbles quickly, and storing those on the web too.  I’m not sure about the Writely word processor and the spreadsheet (whatever that’s called) – when I last used Writely (months ago) it wasn’t very impressive.  But there is little doubt in my mind that for web-oriented applications, and for basic calendar/diary stuff, keeping things on the web and accessing them through a browser is almost ideal, provided I can rely on being ‘always connected’.  Broadband in the UK is becoming more and more like running water: only the remotest places don’t have it.

The bigger question is whether I trust Google with all my data.  How comfortable am I that Google can (in principle, if not in fact) read my emails, see my calendar, know what sites I monitor via RSS/Atom, and read my notes and documents?  Currently, none of this bothers me in the least, for a number of reasons.  First, there are other large companies which already know far more about me (and my lifestyle) than Google does: my bank and the supermarkets I use, for example.  Second, I can choose what I store in any of the Google applications, and I could easily and cheaply encrypt anything I didn’t want stored en clair.  Lastly, I’m pretty sure that anyone looking at my Google data would have a hard time finding anything worth exploiting anyway: I’m not stupid enough to store  anything truly valuable there.  Every day, we all leave behind us trails of transactions, CCTV images, cellphone location updates, building entry/exit events, etc.  These things are (for most of us) completely out of our control: is it paranoia, stupidity or vanity that makes people talk-up the ‘danger’ of online services?

Another long gap between posts.  So what’s happening?  Well, we went back to The Hague the other weekend to visit friends we made while living there.   It was great to have the chance to eat, drink and chat with them again: the children all went to school with each other for five years or so, and despite being apart for over a year they all got on as if they’d never been apart.  On the Saturday eveing we got together at the excellent, friendly Lof der Zotheid, in Breitnerlaan.  It’s evening like this (and food/drink like this) that I do miss.  The next morning, our hosts (Simon and Maureen) laid on breakfast for four families, so the party extended well into Sunday!  Eventually, we had to make our way back to the Hoek, for the ferry.  The journey’s so easy, we ought to do this more.

Daniel had his 13th birthday in July!  We managed to get him what he really wanted: a good, second-hand soprano Saxophone (a Yanagisawa!) from the ever-helpful folk at Wood, Wind and Reed.  Daniel loves it.  Good thing too, because it wasn’t cheap!  But it’s worth every penny, as he will do it justice – he’s already working on Grade 7 (on the Alto Sax) – and he’s playing with great assurance.  As a birthday party, we took half a dozen of his friends paint-balling at Go Ballistic in Thetford Forest.  I must this is a very professionally run place: superb support for the whole day, everyone had fun and no-one got hurt.

Becca has just returned from a week’s camping with her Guides patrol.  They were somewhere near Coltishall, and it did rain for a couple of days but this doesn’t seem to have dampened her enthusiasm; she’s starting to plan a family camping holiday already!  This week, she’s been abseiling, climbing, kayaking, hiking, and on some enormous zip-wire, so coming home is a little bit of an anti-climax for her.

Work-wise, it’s been a busy time, working on a healthcare-related project (more about that another time), leaving little time to get to grips with the personal development projects.   It also means I have plenty to write about my adventures in the world of HL7, DICOM, CDA and the rest of the healthcare informatics alphabet soup.  So much to do, so little time…

I’ve been meaning to use a Wiki to keep notes on projects and as a cheap, fast way to publish stuff to the web, but I don’t write often enough to make it worth investing too much time there.  At work we use the Confluence publishing system which is pretty good: it’s commercial, but there’s a free individual license.  I tried it,  but the (memory) overhead of running Tomcat on my virtual server is a little more than I can accept for just a wiki.

DokuWiki is superb, and with the Monobook theme it looks and feels very much like Mediawiki.  It’s a PHP wiki engine, a much lower-overhead proposition and almost as functional.  My hobby project now is to add a simple XMLRPC interface, which I intend to use as the basis for a page publishing and editing facility, via OpenOffice Writer (and possibly Word, too).   I’m going to use Python on the server-side; it’s a superb language and very good for web development.  I’ll probably use Webware for Python to run the server-side code – it’s a fast, simple and effective container.

Two weeks ago, following some interruptions of service, my hosting provider simply went out of business.  This was a bit of a shock, as RazorLogix (RL) had looked like an almost perfect shared host – great service (until the outages), and a very good shared-host plan which included SSH access (non-root), Java, Python (including mod_python), RubyOnRails, Subversion support and more.

When the end came, it meant backing up everything and moving to a new host. Having done this only a month or two previously (in order to move to RL!), I wasn’t very pleased.  The first problem was, who to host with?  I looked around for something close to the RL offering.  There are other shared-host companies which offer mod_python, but usually these are a bit constraining and very few offer SSH access.

In the end, I decided to take a slightly bolder step and go for a  Linux virtual server.  I hadn’t considered running my own server before because it had meant either running the hardware myself 24×7 (not really something I want to do), or purchasing access to a rack-mounted server (very expensive).  Now that virtualization has arrived in the hosting market, running your own server is very cheap and gives you complete flexibility.  The downside of course is that you have to do everything else: administer the server, install software and so on.

There appear to be two main approaches to the virtual server market: User Mode Linux (UML) or using a virtualization layer such as Xen.  The result, as far as the user is concerned, is broadly the same: you have a complete Linux OS environment, an allocation of main memory, and some disk space (a few GB).  Of all the providers I looked at, the two I concentrated on were Bytemark, which uses UML and Rimuhosting, which uses Xen.

I chose Rimuhosting.  So far, so good.  They are based in New Zealand but most of the hardware seems to be in US datacentres.  Requests for help have been answered promptly and they do seem to know what they’re doing.  The standard VPS install has almost everything I need, right out of the box, including Java and Python. For most of the basic setup, there is a web-based control panel.

As for me, the experience has been a bit like going back in time, to the years I spent developing in C and C++ on Unix (12 years or more ago).  It’s surprising just how much has stayed with me, even csh/bash stuff.  The biggest challenge has been understanding the components of the standard (RedHat Enterprise-based) distribution that Rimuhosting uses, e.g. Postfix and Dovecot for email.  Having SSH access (and SCP access, so you can use something like WinSCP) means complete freedom: now I can do some of the server-side stuff I always wanted to do!