thread dump

The previous couple of posts were related to using Apache Camel in a healthcare integration setting.  The HL7 support in Camel is built on the HAPI library which provides Java classes for most of the HL7 v2 variants, plus MLLP support.  So in integration terms, HL7 can be considered a combination of an endpoint type (the MLLP/TCP protocol) and a message format (the HL7 ER7 or XML representation).

The Camel architecture nicely isolates endpoint-types and message-structure types in Components and DataFormats, respectively.  A Component is something which knows about a specific communication mechanism or protocol: in the case of HL7, the Camel HL7 Component knows how to exploit the underlying Apache Mina library to exchange delimited text messages using the HL7 protocol.  The Component also acts as a factory for Camel Endpoint objects: it hands out individual Endpoint objects, each of which represents an instance of the endpoint-type implemented by the Component. Camel DataFormat objects marshal / unmarshal byte-streams or strings to / from Java objects of some type.  This makes it much easier to build content-based routes in Camel.

HL7 ACK / NAK Messages

In the previous post, one of the issues I had to deal with in the example route was returning an ACK or a NAK to the message originator. The HL7 protocol uses a positive-acknowledgement scheme: receivers will ACKnowledge (ACK) messages they understand and can process, and will Negatively AcKnowledge (NAK) anything else.  The HAPI library provides support for this in the makeACK method of the DefaultApplication class.  Unfortunately, this is less helpful when we need to return a NAK.

Recall that the first part of my route was checking to see whether the right kind of message has been received. In the sample, I was testing just the message type (MSH-9.1) and the trigger event (MSH-9.2) fields.  In reality, you would probably also want to check that the sender was using the same version of HL7 as you are expecting: you can do this by examining the value of field MSH-12. It will become clear why this is important, below.

Suppose the inbound message fails the MSH-9 test. The otherwise() clause in my little route will send the HL7 data (unmarshalled from the stream) to the badMessage() method on my handler bean.  This method expects a single argument of type ca.uhn.hl7v2.model.Message, i.e. a generic HAPI HL7 message object.  The argument can’t be of a more specific type than this because the Camel route builder cannot know in advance what will be received and unmarshalled from the endpoint; it can only assume it will be a HAPI Message.

Constructing an ACK

In the badMessage method, I need to construct a HL7 NAK message to be returned to the caller. HL7 ACKs and NAKs share the same structure: we just need to change the value of one field (and optionally add a diagnostic message) to indicate NAK. The obvious solution is to call DefaultApplication.makeACK, mentioned earlier, and modify the returned structure as necessary.  MakeACK’s single argument is a single Segment, the intention being that the caller passes-in the MSH segment of the original inbound message:

Message makeACK(Segment inboundHeader)

The reason for this is that the makeACK method needs some values from the inbound MSH segment to populate fields in the Message Acknowledgement segment (MSA) of the ACK correctly. An HL7 ACK (or NAK) must be associated with the original message (i.e. the inbound message being ACK’d or NAK’d) so that the originator can distinguish which of its outbound messages is being acknowledged or rejected. This is done using the Message Control ID, which is field 10 of the MSH segment.

But remember where we are in the Camel route: we have an invalid message in our hands. We don’t know the exact class-type of the inbound message object so we can’t cast the Message to a specific type. And the Message interface itself doesn’t provide a way to extract the MSH segment. So how are we going to call makeACK(Segment)?

We cannot simply new-up an instance of some specific message type (e.g. a version 2.3 ORM^O01) and use its MSH segment as the makeACK argument:

    // Assume ca.uhn.hl7v2.model.v23.*
   
    ORM_O01 orm = new ORM_O01();

    ACK ackMsg = (ACK)DefaultApplication.makeACK(orm.getMSH());        // <– Bang!

    // Set MSA.1 to Application Reject (AR)
    ackMsg.getMSA().getAcknowledgementCode().setValue(“AR”);

    // etc.

This will compile, but won’t work. The manufactured ORM_O01, despite being an instance of a specific version of HL7, is not properly constructed, in several ways. First, MSH-12.1 will not contain “2.3″, which is surprising, given that HAPI should be able to populate MSH-12.1 for a new message instance. If we pass the ORM_O01 MSH to makeACK, it tries to get the value of MSH-12.1 but finds a null. This causes makeACK to fall-back to creating and returning an ACK for version 2.4 of HL7 (this is baked-in to the HAPI code – I’m not sure if this is an arbitrary HAPI design decision, or to do with the HL7 spec). And of course we get a ClassCastException at run-time as a result, because we are expecting a v2.3 object.

But even if this did work, isn’t it ugly? We’d be baking-in a version of HL7 because we must choose a specific ORM_O01. So how do we create the ACK structure, and how can we make our code work for any version of HL7 supported by HAPI?

Terser to the rescue

Fortunately, HAPI provides a handy utility class for parsing and unparsing HL7 messages of arbitrary versions – the Terser. This will let us get hold of the MSH segment from the original message, so we could at least call makeACK with the original MSH:

    // This still depends on the ackMsg decl. being for the same specific version
    // as the original message.
    Terser t = new Terser(originalMsg);
    Segment msh = t.getSegment(“MSH”);
    ackMsg = (ACK)DefaultApplication.makeACK(msh);
    ackMsg.getMSA().getAcknowledgementCode().setValue(“AR”);

Now when we call makeACK we are passing-in a valid MSH so it will extract the HL7 version number and dynamically create the correct ACK class, using Class.forName.  But as the comment says, this stilll only works if the compiled-in ACK declaration matches the returned ACK message version. We need to lose the compiled-in ACK declaration, but still find a way to populate the ACK message’s MSA segment with the right values.

Fortunately, we can use the Terser to set the field values in a version-independent way. This is exactly what DefaultApplication’s static methods makeACK and fillResponseHeader do.  So I used a combination of the Terser and the guts of DefaultApplication.makeACK to create a couple of helper methods which do the right thing.  I’ve put the source code on this page in my Google site, for anyone who wants to use it.

One of the healthcare solutions I’m working on has to generate diagnostic report documents in a form we can distribute directly (e.g. secure email) to clinical staff. The diagnostic report text plus the usual patient demographics and such, arrives in a custom XML message from the integration engine. The solution uses HL7 v2 elsewhere, but the PAS messaging is exclusively via custom XML (not HL7 v3 XML).

The legacy document-generation subsystem used Word to create the document: a template was created to get the layout and formatting right, and simple placeholders used to mark the locations of the data we would substitute, taken from the XML data. This approach actually loads Word into memory (on the server!) to do the work – it’s slow, memory-hungry and just generally clumsy and ugly. Plus you end up with a Word document, so the clinician needs Word (or the reader) to view it.

PDF is a more acceptable format, in my view. We can secure and digitally sign the document when we generate it to prevent subsequent changes. Recipients can view PDF on any platform with a free viewer. The problem for me was how to generate the PDF programmatically, from the XML data. There are probably several ways to do this, but I chose XSL-FO and the Apache FOP project mainly because I wanted to avoid using a proprietary PDF generator product (there are lots out there), but also since XSL-FO can do more than just generate PDF.

First problem: how do you create the FO which is the ’shape’ of the generated document? Of course, you could simply read through all the manuals and write one from scratch. Well, I’m just plain lazy, you see, and I don’t want to do all that. I want to take my nice OpenOffice document, or even Word document, and have a tool create an XSL-FO for that document. And I’d like the tool to be free (there are commercial tools of course, but I’m cheap). Does such a thing exist?

It does! Amazingly, precisely this facility exists in Abiword: not my favourite word-processor by a long, long way, but a good solution for this particular problem. OpenOffice should be really good at doing this, as it stores documents natively as XML and already uses FO internally for some style information. But, despite some promising hints, there is no mature support for this. This is a real shame: this is just the sort of thing OOo should be capable of, especially as it’s apparently half way there already.

Here’s what I managed to find on the OOo site:

• Some OpenOffice transforms and things.
• And this one talks about ooo2xslfo but that project looks as if it’s just a SVN repository – not sure I have time to play with that.
• The OOo XML site has a page on XML filters too.
  

Important to mention also that Microsoft does have an XSLT which you can apply to Word documents to generate XSL-FO. It’s freely downloadable from this download page. I tried this, and it works, but the resulting FO is much messier than Abiword’s.

Once you have the FO, the obvious step is to embed it in an XSL, add xsl:value-of elements in the appropriate places and use a transform to populate the template. This is the approach I took for the proof-of-concept and it worked well. The resulting PDF looks almost right – with a small amount of FO-tweaking, we should have something very usable.

But using XSL means loading up and running the (trivial) transform which I think may be very inefficient for such a simple case, plus it requires the FO to be edited. I’ve decided to use a simpler approach (using StringTemplate) which I hope will be more efficient, and requires less FO editing (just the addition of $fieldName$ placeholders). All we need is a list of (fieldname, XPath) pairs for our XML message, in order to drive this template.  Of course, most other applications will need the power of XSL (e.g. to deal with tables of entries): I’m only avoiding it here because the data is so simple.

This is something we’re bound to want to do again, in different contexts, so I’m using this project and the prototype to build a tool-chain and utilities for this, so we can use the same approach more easily next time.