Innovative open source software

This post was originally published on AIIM's Expert Blogs by Serge Huber, CTO at Jahia Solutions

********

When I originally discovered open source software, most of the projects were usually works of love, developed by passionate amateurs in their free time and that were interesting but were often not production ready, poorly documented and sometimes less innovative than closed source software. In other words, open source software was, at the time, mostly a cheaper, less powerful alternative to mainstream established software mostly considered because of budget constraints. Today the opposite may be said of some projects, and this article will illustrate this with a few examples.

The first example is of course the best known one : Linux. Initially the Linux kernel was started as a hobby project by Linus Torvalds, and it didn’t have much pretense except to help him learn the intricacies of operating system kernel development. When the project was started, it was nothing innovative, it was definitely not production ready, and not even innovative since it was mostly implementing already existing algorithms.

Fast-forward to today: now Linux is used everywhere from large clusters to mobile devices, and the same kernel has incredible flexibility. It is maintained and contributed to by many different commercial companies, both small and large, and it is definitely reliable, production ready and well documented. What about innovation ? While a lot of the original design still remains, it has among the best implementations of input/output locks, and has been at the forefront of experimenting with new mobile, storage and cloud technologies. Therefore, despite its very modest debut, this is a very good example of an open source project that has truly realized its potential. It took quite some time for Linux to become such a successful projects that can seriously compete and even beat proprietary solutions, but the world is now a different place, where open source solutions are not only readily available but also among the most interesting choices.

While we’re talking about the cloud, among the new trends are of course Big Data. Big Data requires new frameworks to make it easy to install and maintain large number of cluster nodes, and easily distribute software workload on these systems. Again, open source projects such as Apache Hadoop and memcached have been helping companies deliver solutions that are production ready and scale well to thousands of nodes, and are entirely based on the efforts of the open source community. Even though their origins may have been commercial, these projects have been contributed to the open source world, and have greatly benefited from other contributors that have since rallied the ranks of the initial developers. Another interesting cloud project worth mentioning is OpenStack, which helps set up, install and administer a cloud infrastructure. As a testimony to how good these projects are, the list of Hadoop users is nothing short of amazing, and is probably not even up to date !

Closer to the content world are projects such as Apache Jackrabbit, also originally contributed  by a commercial company, but that is now used as either parts of a software product or simply as a custom content back-end. Again, it makes a lot of sense to share code at the infrastructure level, since competition may still happen at the higher levels of the software, while the infrastructure code may be expensive to maintain on your own. The Jackrabbit project is also at the origin of the Java Content Repository API (JCR) specification, here again an innovation that came from the open source world.

One of the most powerful open source search engines, Apache Lucene, also has a brother project called Apache Solr. Solr is actually a search server based on Lucene that also adds powerful features such as XML/HTTP and JSON/Python/Ruby APIs, hit highlighting, faceted search, synonyms, spacial search, search by semantic proximity, caching, replication, and a web admin interface. Solr also supports distributed searches across multiple hosts, again leveraging clustering technologies to build powerful solutions. This project is a good example of a specialized content project that has many applications throughout the industry and that is very innovative.

An open source community I recently became familiar with is the Open Participation Software for Java (OPS4j). The idea behind this project is that open source software participation should be as open as participation in the Wikipedia project is open, meaning that you don’t have to go through any vetting or election process to be accepted as a contributor on an open source project. Of course you might wonder how quality may be achieved through such an open model, but it actually works, since it is based on infrastructure that can withstand malicious activity such as versioning for rollback, backups, unit tests, etc. I definitely think that this “new” open source model is interesting, and I will be following it closely (and actually already committed a minor bug fix :)).

Open source software has also almost become the norm in new standards implementation. This trend is relatively recent, but if you look at standards such as the JCR or CMIS, both were immediately implemented as a freely available open source project. I think that this makes a lot of sense since a standard implementation is similar to infrastructure work, and this makes it easy for companies involved in the standard to share their work and improve the overall quality and availability of the new standard. For interoperability standards such as CMIS this is especially useful since it can really help with the hardest work of making sure that different implementations actually do work together and that issues are raised and fixed quickly and transparently (bypassing corporate politics). It is therefore also my hope that the new upcoming Web Experience Management Interoperability standard will also have an open source implementation.

While open source software is really interesting because of the immediate benefits of software quality or innovation, it is also interesting as a long to very long term solution. Companies may come and go, but open source software (provided the source is never lost), will remain available whatever happens, and this is really reassuring to all the actors of the computing world (especially those who still have to maintain closed source Cobol software). Because of this, sometimes the open source solution is the best choice even if it is not (yet) up to par with a commercial solution, since if it is planned to stay in place for a long time, it will definitely withstand the test of time. So for the new year, you should definitely, if you haven’t already, consider open source software seriously when selecting a solution, as failing to do so you might actually be missing out not only on budget reductions but also innovation.