OpenMAMA 2.3.0

We released OpenMAMA 2.3.0. It's awesome, and has loads of shiny things.

It's been a long time coming, but today we're finally able to announce the release of OpenMAMA 2.3.0. So what's changed in this release, and why the long wait? Let me respond to the first point, and then hopefully the second should become a bit clearer.

For a start, we now have a complete open source stack, with a market data capable publisher (capturereplayc) able to replay actual market data (stored as MAMA playback files). This data includes the full compliment of data types, including using OpenMAMA's vector messages for transmitting order book data. And it works with OpenMAMDA, so you can see how we can easily handle streaming order books and other complex data types, right within the API.

We've also supplied other tools to complement capturereplay - capturec, which enables the recording of data published by an OpenMAMA source into the MAMA playback format; and captureconvertc, which allows you to easily convert playbacks between different payload types. Written a bridge and want to check it works using data captured from other environments? Check.

But how do we get this data around? We've also built, with the assistance of a few folks over at RedHat, a new open source middleware bridge based on the Qpid Proton AMQP implementation. The Qpid bridge is designed to be a fully featured reference implementation of an OpenMAMA bridge, including access to the majority of OpenMAMA's functionality. The code base provides extensive comments around the bridge interface, providing developers with a simple reference when looking to understand how the interface behaves. Using the OpenMAMA common timer and queue implementations means the bridge also demonstrates how other middleware's can make use of common functionality when it is not provided natively.

The payload bridge is also complete, providing full support for complex data types, including MamaPrice, MamaDatetime, and vector types, including vector messages. This level of support allows Qpid to provide a full stream of data into not only OpenMAMA, but also the OpenMAMDA layer, including full depth orderbooks.

Obviously the efforts to develop a fully open and functioning stack have been a big focus, but we've also spent a lot of time trying to make it easier for developers to contribute to OpenMAMA. This effort comes in two forms - engagement, and tooling. From an engagement stand point, we've made significant improvements in our documentation, updating the OpenMAMA wiki with masses of useful information. We've also attempted to improve our processes, so we can review patches faster, respond to questions more effectively, and generally engage more with the community as a whole. The response to our IRC meetings for example was phenomenal, and it was great to see so many interested parties together for the discussion.

From a tooling perspective, we've completely replaced the existing build infrastructure with a new SCons based system. SCons really is a next gen build system, with a lot of very interesting functionality, and we've been lucky to leverage the skills of the NYSE Technologies Build and Release team for a lot of that work. Having a unified build system that works on both Windows and Linux platforms has made the project worthwhile in itself, but the flexibility of SCons should allow us to make further improvements in the future.

Supplementary to this was the project to open source a complete set of unit tests for OpenMAMA. This project was also completed late last year, with NYSE Technologies engineering again providing us with a substantial baseline. Following some refactoring and improvement work, we're now left with a test suite of over 1500 individual tests, covering the majority of the OpenMAMA C API, including a full set of tests for the middleware bridge interface. This again provides a significant utility for other bridge developers, who can use the unit tests to determine exactly the sort of behaviour which is expected from various OpenMAMA calls, both internally and externally. Indeed the unit tests were particularly useful during the development of the Qpid bridge mentioned above.

So what else? Well, we've had pretty extensive contributions from a number of sources, but I was saving full details for a later blog post. Suffice to say the current release is the result of over 200 patches, and 50,000 code line changes. Hopefully at this stage it becomes clear why we took so long getting this one together. We've been very busy trying to make OpenMAMA an easier API to work with: easier to build, easier to install, easier to run, easier to develop against, easier to contribute to. We're a bigger community than we were a year ago, and we'll most certainly be a lot bigger again in six months time. We still have a lot more work to do, but I think that'll do for the current release...