On the missing plugins, you can get access to the missing XML Schema Editor plugin by adding a new update center:
- Name: NetBeans Dev
- URL: http://updates.netbeans.org/netbeans/updates/6.8/uc/m1/dev/catalog.xml.gz
But when I tried to install it from here, I got this error:

So this might be the reason they left it out of 6.8, but it would have been good to be warned and even better if a fix had been promised.  My cynical side wonders whether this is some sort of stealthy de-scoping of ‘enterprise’ features, coinciding eerily with progress on the Oracle/Sun deal and a less then fulsome commitment from Oracle on the future of Netbeans.  I do hope I’m wrong, but other folk clearly think that Oracle’s view of NB is ambivalent at best.

Meanwhile, back to more practical things Netbeans-related.  Have you recently updated your JDK installation (to JDK 6 Update 17)?  What happened when you next ran Netbeans?  Did you see a dialog telling you NB couldn’t find the JDK?

Annoyingly, when Netbeans installs itself, it creates an entry in the netbeans.conf file (in C:\Program Files\NetBeans 6.7\etc or the equivalent path in your installation) containing the explicit JDK path, e.g.

  # Default location of JDK, can be overridden by using –jdkhome <dir>:
  netbeans_jdkhome="C:\Program Files\Java\jdk1.6.0_17"

It’s seems a small thing, but forcing a user to fix this by locating and manually editing a configuration file is desperately poor.  As NB makes this check as it starts up, it should make more of an effort to detect alternative JDK installations, present the user with what it finds and offer to adjust the configuration for you.  It wouldn’t be hard.

This was started some time ago by Bryan Tripp at the University Health Network in Canada, but recently the work of maintaining the project and pushing forward with development has been taken on by James Agnew.

Version 0.6 brings support for HL7 v2.5.1 and v2.6, and separate jars for the different HL7 versions. James’ announcement also hinted he’s looking at significant performance improvements in future releases. This is great news: thanks are due to James for his continued good work.

If you’re trying to find the bits, note that the download link isn’t in the HAPI site sidebar – go to the Using HAPI / Developing page, and you’ll find the link to the Sourceforge download page there.

The past few years have been interesting and fruitful from a programming language perspective: Ruby, Groovy, Scala, F#, and the evolution of C# are those which spring to mind.

Many of these developments have been about finding a way to push functional-programming idioms (lambda expressions, currying) or dynamic language features (type inference, closures) into well-established imperative languages like C# and Java. I think the results have been mixed: imperative languages are essentially assignment-based, not expression-based, so you end up with a hybrid which, from a language syntax point of view, is neither one thing nor another and arguably suffers as a result.  But it’s difficult to argue that these efforts don’t deliver real positive utility.

An example of this is how C# has evolved support for lambda expressions, expression trees and implicit typing. First (in v1) we had simple delegates, then (in v2) we were able to drop the delegate syntax and declare anonymous methods inline, and finally (in v3) we can use a compact lambda syntax which looks a bit like ML. But because all this has been added to what remains fundamentally a statically-typed, compiled language, you still have to put up with all the nasty explicit declaration of types (and type parameters) on the l.h.s. of every assignment, just to keep the compiler happy.  So the final step was to introduce the var keyword and bring implicit typing into the language. Not dynamic typing, just implicit typing: we get to trim away the ugly declarations but keep the compile-time type safety. 

The confluence of these developments enabled things like LINQ: blending a query sublanguage into the primary language syntax. The best exposition of this I have read is by Ian Griffiths, in his excellent post on expression trees: see the example code just over half-way down, under the section ‘Expression Trees, LINQ, Deferred Queries, and Databases’. It’s all so superficially appealing – just look at the motivating use-case in Ian’s post. But still there’s a small, nagging voice in my head saying this isn’t quite right…

It feels as if we’re trying to pretend there’s no boundary (or intermediary) between the program text and the resource we’re querying. This is similar to the way we tried to pretend that remote procedure calls were really local calls in the old (and now largely discredited) distributed programming models such as CORBA and DCOM. Recall the fallacies of distributed computing and the tenets of SOA: these are essentially warnings about the dangers of implicit or flawed assumptions.  Although the tenets have been challenged (and were threatened with retirement) most of what they said is timelessly valid.  In the context of LINQ, I think my nagging voice is pointing to the boundary-crossing in a LINQ query and muttering “private implementation technique, not primitive construct”.

What’s this got to do with my starting point for this piece, which was Harold’s ‘Edsel’ jibe?  I suppose it’s this: whatever we may feel about the virtue of purity in language design, in the end productivity and usability are the bigger drivers, and with skill it is possible to combine language features without creating a monster.

An editorial in SD Times sums up Sun’s difficulties nicely as ‘too much vision, too little execution’.  It’s not enough to give away tools and hope the next generation will adopt them. You need to innovate and to lead, because the brightest and best of the next generation want to climb the mountain, not travel for free in the foothills. To innovate, you need great scientists and engineers; to retain those folk, you need to pay them adequately and provide them with the facilities and resources they need to be brilliant. To do all of that, you need money.  And that comes from sales.  It’s as simple as that.

Just look at Microsoft Research: you just can’t foster this quantity and quality of innovation unless you have a lot of money to invest, and you only get that by doing a lot of very effective monetizing.  Microsoft and Oracle: we don’t like them very much, do we? But they’re really good at monetizing.

As for Java, most seem to agree there is no threat to its future as an enterprise platform: it’s open-technology now and well embedded in the enterprise.  What about the future of the Java language? Do we really need to worry about that? The language itself matters much less than the JVM, which surely has a very bright future given the newer languages targeting it, especially Scala and Groovy.  Should we care whether (e.g.) closures make it into the Java language, if we can use newer languages like Scala and continue to leverage all the existing Java libraries, components and infrastructure?

But I have a soft spot for Sun. My first proper programming job involved writing C/C++ (and using Cfront – remember that?) on a Sun-3 workstation, and various versions were part of my working life for some time. I loved the solid feel of these machines. Remember the optical mouse that only worked on those special, shiny metal mats?

Between then and now, my only links to Sun have been through OpenOffice and Java. I played with Java quite early on, abandoned it in favour of Microsoft .NET, but have recently (and happily) returned to it. For the last couple of years I’ve been an enthusiastic user of NetBeans – I do hope Oracle recognizes just how good NetBeans is.  I have also tried OpenSolaris: ZFS is simply awe-inspiring, and very nearly enough on its own to make me run OpenSolaris, though truthfully I don’t think I need ZFS, and Windows remains simply more convenient and usable for everyday.

I’ve used OpenOffice for a long, long time.  I know Writer pretty well, warts and all. You need to ignore some of the cosmetic shortcomings, persevere with it and appreciate its fundamental strengths; things which I think make Writer better than Word. Occasionally I use OOo Writer to help colleagues debug and rescue Word documents which have evolved uncontrollable formatting: it amuses me to be using a free tool to clean-up after a rather expensive one.  I really hope Oracle will resource and manage the OpenOffice program properly. With the right additional effort, they have a potential Office-beater.

But the crown-jewels are Java itself, and the NetBeans IDE.  Everyone is trying to second-guess what Oracle will do with Java: I don’t have anything to add.  But I really want to add my voice to those hoping Oracle will recognize just how good NetBeans has become, not just as the best Java IDE out there, but also a first-class platform for building rich-clients.

This is great for those who have a copy of VISIO (or whose employer pays for a copy), but for those of us who don’t want to pay however many hundreds of dollars Microsoft wants for VISIO these days, an alternative (especially a free one) is always welcome.  OpenOffice (OOo) Draw, though hardly as functional or attractive as VISIO, is both free and good enough for many diagrams.

So I decided to see if I could create a set of OOo Draw shapes from the VISIO stencil, and see whether Draw could be practical for integration diagramming.  In short, I succeeded in converting the stencil, and I think Draw is just about good enough. For those not interested in the background, I’ve made the material freely available from this page on my website, so just go ahead and download/install.  Otherwise, read on.

Exporting the VISIO Shapes

One way to get shape data into Draw is via SVG. So I started with the VISIO shapes, freely downloadable from the EIP site, and as I use VISIO at work I was able to load up the stencil and export it as SVG. 

You need to install an OpenOffice extension for importing SVG. I’m using OOo 3.0: the extension installed without incident and the import was straightforward. I was surprised at how good the resulting shapes looked in Draw.  Each shape is composed of drawing primitives, so you need to select and group them to create a shape you can place and move easily. I’ve done this for all the shapes in the download.

So now we have the shapes, where can we put them to make them easily reusable?  In Draw, there’s no stencil feature as such: the only option is the Gallery:

The Gallery

The gallery feature in Draw isn’t very appealing, initially. When you first install OOo, the gallery contains some spectacularly awful materials: nasty backgrounds and bullets which I cannot imagine ever wanting to use, and it even contains some pointless sound-clips that surely only a 5-year old might use (and only then if they were desperate). The shapes and graphics are reminiscent of shareware software from the Windows 95 era.  And what is the point of the built-in media player?  Why can’t OpenOffice grow up?  It’s so very close to being excellent in many respects: things like this just make it look silly and dated.

Anyway, back to the gallery feature. The one saving grace of this feature is that you can create new gallery themes and add your own shapes to them. To do this, create a new theme, give it a name and then drag your shapes into it.  Note that when you drag/drop your shape you must hold down the mouse button and pause a second or two before dragging: this pause changes the drag-mode from a simple shape move to the required move/copy.

I’ve done this for all the EIP shapes, creating an EIP gallery theme:

Sharing the EIP Theme

Unfortunately, Draw doesn’t implement a theme export facility so to share this new theme with others requires some irritating file copying.  The contents of the Gallery are stored in your profile (or home) folder. On Windows this is likely to be:

C:\Documents and Settings\<user>\Application Data\OpenOffice.org\3\user\gallery

On Unix/Linux, check your home folder. There are several file types in here and (unusually for OOo) they are binary, not XML. This makes it difficult to reverse-engineer their purpose and relationship but fortunately it doesn’t seem to be necessary to understand fully what’s going on. The key thing to note is that the numbers in the file names bind together related files.  When you create a new theme, a new set of files appears to be created.

For the EIP theme I have manually renamed the files to use numbers which I hope won’t clash with any existing gallery files you may have.  Check the zip contents before you extract the files into your OOo folder and if you have existing files with the same name, simply rename the files in my zip to be unique in your environment. 

The Result

I think the result is pretty good.  As a test, I have (partially) reproduced the diagram from the end-cover of the EIP book in a Draw file (.odg) which is included in the download zip:

I hope this may be some use to other folk.  Download from this page.

The best solution is simply to download the complete NetBeans/GlassFish/OpenESB combination from the OpenESB downloads page and work with that. On the download page the top section contains the GA (stable) GlassFish ESB / NetBeans combination, and the nightly build of the latest (unstable) additional components (i.e. binding components, service engines etc.) which match this GA release.

The next major section on the page contains the GlassFish ESB v2.1 downloads (milestone 1 at the time of writing), which are not yet considered stable. I grabbed the 2.1 release, plus the HL7 BC (and NB design-time modules) from here.  Installation was completely pain-free.  I particularly liked the way the additional components are packaged as a jar-based installer – very nicely done. 

Because the main package is a GlassFish ESB bundle, it installs by default to a non-standard path: C:\GlassFishESB. Everything is installed under this directory, including the configuration files for NetBeans, which would normally be placed in your home directory (under .netbeans). Presumably this is done to allow parallel installation with a stable NetBeans/GF without either interfering with the other. What this means, of course, is that all your preferences (editor settings etc.) are not transferred. This is easily fixed, manually, by copying over the stuff from: C:\Documents and Settings\<user>\.netbeans\6.5\config\Editors to C:\GlassFishESB\.netbeans\glassfishesb\config\Editors

Anyway, back to the BPEL issue.  A clean rebuild of everything was sufficient: the CA deployed to GF, and it ran fine.  I used Hermes JMS and watched the test messages arrive on the queue. Now I can push ahead and explore more of OpenESB and BPEL.

The BPEL is trivial, and the BPEL copy elements for optional elements in the source schema are marked with the ignoreMissingFromData attribute.  I have rebuilt everything from scratch.  I know the redeployments are successful because the error reports points to the correct line following an edit and I can use the BPEL debugger to step through the assign, eventually failing on the optional element.

I was given the hint about the ignoreMissingFromData attribute by Michael Czapski, in a blog comment.  Googling around for more help, I came across a few other postings, and discovered that the attribute could be placed at the process level or at the individual copy activity level. Jeff Sexton also has a nice post on the assign issue.

In Michael’s tutorial he does in fact place the attribute at the process level (see p.95 of the PDF document which accompanies his tutorial). Placing it at the process level is not very attractive because it will mask the absence of mandatory fields.

When placed at the copy element level, it seems to fail. Here’s the error I see, as output in the GlassFish console window:

BPCOR-6151:The process instance has been terminated because a fault was not handled; Fault Name is {http://docs.oasis-open.org/wsbpel/2.0/process/executable}selectionFailure; Fault Data is null
com.sun.jbi.engine.bpel.core.bpel.exception.StandardException: I18N: BPCOR-3023: Selection Failure occurred in BPEL({http://enterprise.netbeans.org/bpel/HL7Processor/bpHL7Processor}bpHL7Processor) at line 28!
BPCOR-6129:Line Number is 26
BPCOR-6130:Activity Name is Assign1

Note the line numbers in the above report: the first of these (line 28) is where the selection failure actually occurred, line 26 is the opening tag of the enclosing assign activity element.  Here’s a shot of the source so you can see the corresponding line numbers. I placed all the optional elements together at the start of the assign:

So why doesn’t this work? Is this a bug? Perhaps I’ve run into something which only works in the JavaCAPS product and not in OpenESB?  Shame.

Another unfortunate discovery is that the OpenESB UI in NetBeans 6.5 doesn’t seem to support adding the ignoreMissingFromData at the process level.  In Michael’s PDF he selects the top-level BPEL process scope and uses the property panel to set this attribute.  It doesn’t appear to exist in NB 6.5. In the shot below I’ve tried to capture the same UI as in Michael’s tutorial:

Note there’s nothing below the ‘Documentation’ entry: presumably this is additional UI support which you get in the commercial JCAPS product. I’m not that concerned with the missing UI, as long as I can add the attribute in the XML source.  So I edited the BPEL XML, using Michael’s tutorial source as a guide. It’s easy to add the process-level attribute; only the namespace prefix had to be changed:

<?xml version="1.0" encoding="UTF-8"?>
<process
    name="bpHL7Processor"
    xmlns:ns3="http://www.sun.com/wsbpel/2.0/process/executable/SUNExtension"
    ns3:ignoreMissingFromData="yes"
    targetNamespace="http://enterprise.netbeans.org/bpel/HL7Processor/bpHL7Processor"
    xmlns="http://docs.oasis-open.org/wsbpel/2.0/process/executable"

So another clean/build/deploy and … it still didn’t work. So now I’m stuck, and I hate being stuck. It’s almost enough to put me off the product.

The impression I get is that BPEL is great for very simple maps and transforms, but anything complex quickly becomes awkward. I haven’t gone beyond ‘simple’ so far, so I’m getting a little concerned. The pretty visual tools and NetBeans integration are cool, but the complexity overhead seems high. It reminds me of Microsoft’s BizTalk product from a few years back – a lot of fiddly config, and not being able to ‘see the wood for the trees’. 

But I want to persevere with OpenESB: something about this product suggests that you need to reach a certain level of enlightenment, after which everything becomes clear and the benefits outweigh the overheads. I’m just not there yet.

I am a developer of another online backup product powered by Amazon S3 http://cloudberrydrive.com/ that we are going to release to beta soon. I would appreciate if you take a look and may be post a review on your blog.

Well I did take a look, but the CloudBerry product isn’t really what I’m looking for.  If all you want/need is a Windows-only client for accessing your S3 buckets directly and copying/moving content, then this product might well be just right.  But it doesn’t really compete with DropBox or JungleDisk.  Andy: if your product were closer to DropBox, and cross-platform, then I would have reviewed it.  As it is, I can’t pass judgement.

If you have time to plough through a very long blog thread, check out Jeremy Zawodny’s excellent piece on Amazon S3 tools. The discussion thread contains links to most (maybe all) the tools out there.

I’ve also been reading the DropBox forums. It’s good to see plenty of other people have requested features I miss (e.g. ability to configure DB to ignore certain file patterns).  There is an upcoming features wiki page which lists some things coming ‘soon’.  Nothing on using your own S3 space though.

JungleDisk attracts me mostly because I can use my existing Amazon S3 storage behind the JungleDisk tool and pay only for the space I actually use. (Note, JD now also uses RackSpace CloudFiles, which actually looks even better than S3).  DropBox is free for 2GB, then a hefty 99 USD annually, for 50GB.  Round one to JungleDisk, in my view.

Then we come to the user-interface. Both tools integrate with Windows and Linux, JD using drive mapping to expose the storage and DropBox using a special folder inside My Documents, with icon overlays to indicate file status.  Both support drag/drop access and run a small tray-resident UI application.

But DropBox is just so, so much nicer to use than JD in the everyday Windows context.  It feels better integrated and the UI seems cleaner.  Other folk have blogged about this difference and I must concur – DropBox has the edge.

Now to the subject which prompted this post in the first place. Neither of these products appears to handle proxy servers particularly well, especially when switching between proxy / no-proxy.  If I restart Windows and forget to switch off the proxy in JD, here’s the mess I’m greeted with when Windows starts:

Yuk. Can’t it simply notice that the proxy isn’t responding, log the fact / decorate the tray icon, and leave it for me to sort out?  It gets worse: if I click on the links (for more information) look what I get:

Is this really what they want the user to see?  This is awful.

DropBox is slightly better, but still doesn’t work properly if I leave the proxy on, and restart. No nasty dialogs, but the network connection isn’t resolved, even if I set it to ‘auto-detect proxy settings’ which according to the DropBox site should use the IE settings. Why can’t these tools auto-detect proxies properly?

DropBox files are cached on the local machine which means if the network is down I can still work on all my files locally, and re-sync when I next connect.  JungleDisk does cache your files, but in a pretty inaccessible way in your profile.  The path will be something like C:\Documents and Settings\<user>\Application Data\JungleDisk\cache\e9998872111157539d8880eca4456345-default

Another good feature of DropBox which isn’t available in JungleDisk is sharing files and folders: in JD, everything is private.

DropBox gets so many things right. The one and only feature I want from JungleDisk is the S3 / CloudFiles backing store. Obviously, the DropBox business model is built around the 99 USD annual charge so I don’t know whether this can/will ever happen.