Sometimes the open core functionality ceiling gets lower

First of all a little bit of background will make the post a little bit more understandable to non I.T. folks.

A bit of background

Zenoss is a network management software vendor with an open source core product, called Zenoss Core, and a closed source product called Zenoss Enterprise.

Zenoss is written in the Python programming language and uses the Zope web application framework.

Relstorage is a Zope add-on for saving data to a relational database. Relstorage allows Zenoss to use a relational database as its storage backend, with all of the scaling out benefits that entails.

A relational database does not need to run on the same server as Zope, you can run it on a completely different server. In fact, you can run the relational database on a cluster of machines giving substantial scalability benefits. With Zope’s native database format, running it on a different machine isn’t possible.

Zope’s native database format limits how scalable Zope can be, which in turn limits how scalable Zenoss can be.

A bit more background

Way back in 2010 myself and others suggested that an open core strategy would lead to some difficult decisions about which features go into the open source product and the closed enterprise product.

I suggested that a feature ceiling could be reached in the open source product and offered some modest proof that it existed.

The ceiling fell in… a bit

Phew, now we’ve got the introductions over, what the heck has all of the above got to do with open core software and lowering the functionality ceiling?

The following is part of a conversation on the dev IRC channel run by Zenoss.

rmatte:

“… A decision was made some time ago to move from standard zopedb to relstorage to improve performance.  Recently, a decision was made to remove the relstorage code from Zenoss Core.”

Bill Karpovitch Co-founder and CEO of Zenoss Inc.

“a mistake we made in the process was that product management had it slated for an enterprise feature but we began development in the core trunk.  for better or worse, our decision was to pull it back. we are continuing to look this as part of the plan forward.”

“all good points.  to be clear, the Core/Enterprise feature decisions are challenging and involve tough trade offs.”

As Bill Karpovitch says above, the relstorage feature wasn’t intended to go into Core, it was supposed to be an Enterprise only feature.

Unfortunately, it kind of snuck into Core accidentally and was removed. This upset the Zenoss community quite a lot as they believed that it would effectively create a fork between the Core and Enterprise products. A fork would make creating add-ons more difficult because potentially Core and Enterprise would need to be tested seperately. Supporting both would also be more difficult due to the core of the Enterprise product being different from Zenoss Core.

The relstorage feature was released in Zenoss Enterprise version 4.1.

Conclusion

BTW this post is in no way an attack on Zenoss, or their very fine business. Kudos to them for having this kind of discussion in public. The above is just a concrete example of the friction that is inevitable when you have an open source product and a closed source product.

The friction isn’t confined to open core companies either, closed source companies have exactly the same friction when they have tiered products too.

There is a happy ending, relstorage was eventually added to Zenoss Core version 4.2.

Network management’s “new wave” six years on

How time flies.

It has been six years since I wrote about Network management’s “new wave” and thought it would be interesting to go back and see what has happened. We are now at the outer envelope of the VC funding cycle so things should be sorting themselves out one way or another.

The “new wave” was Hyperic, Zenoss and Groundwork Open Source VC funded, open source network management companies.

Open source wasn’t new to the network management scene in 2007, there had been well known projects, like Nagios, MRTG and OpenNMS, around for a number of years prior to that.

What was different was combining open source licensing with big wads of venture capital. A total of $79.2M has been invested into the “new wave” over the last 9 years.

What has been the effect on network management software of the combinaton of open source licensing and oodles of venture capital?

Current state of play

My first impression is that not much has changed. Let’s dig a little deeper and see.

Hyperic was founded in 2004 and purchased by SpringSource in 2009 after having raised a total of $9.9M in two rounds of fund raising.

Zenoss was founded in 2005 and has raised a total of $40.8M in three rounds over the last eight years, the most recent being in September 2012 when Zenoss raised a further $25M.

Groundwork Open Source was founded in 2004 and has raised a total of $28.5M in four rounds, the most recent being in October 2009 when Groundwork raised $5M.

Are they still open

Looks like Groundwork isn’t that open. Groundwork Monitor Core product is restricted to 50 devices. The license doesn’t look at all open.

The open source moniker has gone. Hard to tell that, say Zenoss for instance, is actually open source by looking at their home page. If you were an alien just off the mother ship and only had the home page to look at, you wouldn’t know that the core product is open source.

Effects upon closed source competitors

I suggested that the “new wave” would have the effect of opening up the “big 4”. I can see no evidence of this at all. I also thought that the “big 4” would be good candidates to buy the “new wave” and that hasn’t happened either.

Effects upon consumers

The one big winner has been users. Open source network management software ten years ago could be hard work with no proper packaging and woeful documentation. Now, there are some really nice options that are much easier to work with. There are also large communities as well to offer support and guidance where necessary.

Conclusion

I find it hard to believe that too many people would consider the “new wave” experiment to be a major success story. I’m not saying it has failed, but venture capitalists invest money to win big, and the investment hasn’t won big. It probably didn’t help that the financial meltdown happened.

There are a number of winners, not least among the many users who have high quality software to use at minimal cost.

I doubt that venture capitalists will be rushing to find their cheque books to fund another round of open source network management companies.

Had the same wave of money been invested in closed source companies doing the same thing I’d bet that they would have been more successful in strictly money terms at least. If a user jumps on board your ecosystem for the sole reason they can get your core offering for free, is that user going to be worth the same in the long term as a customer who literally bought into the ecosystem? My expectation is that they are not.

I am not saying that open source businesses aren’t perfectly viable businesses. It just means they may not be as profitable as an equivalent closed source business. And money in the end of the day is what venture capitalists are interested in.

Why Monitoring Sucks

Found an interesting old post by John E. Vincent, Why Monitoring Sucks tweeted by MonkChips. What is interesting is what John did next. He created a GitHub account so that he could collaborate with people to rectify the problem.

The most interesting part (to me anyway) is the tools-repos repository in which all of the different monitoring tools are listed.

Enjoy! 🙂

PS: as a counter point, read this post entitled #monitoringlove – a true story by Ulf Månsson.

Musings upon the open core functionality ceiling

One of the things you’d expect from an active open source project is that the code base is likely to grow as more and more features are added.

In An exploration of open core licensing in network management I mentioned that one possible side effect of open core software is the creation of a functionality ceiling.

A functionality ceiling is a level of functionality beyond which the community edition product manager is unwilling to implement because of the fear that the enterprise product will be less attractive to potential customers.

That got me thinking, if a functionality ceiling does exist, how can I demonstrate it?

The graphs below are taken from the Ohloh open source project directory. The rather useful thing about Ohloh is, in addition to cataloguing open source projects, it also performs  extensive code analysis.

The two graphs below are taken from the Hyperic code analysis and the Zenoss code analysis pages on Ohloh.

Hyperic Code Analysis Graph
Hyperic Code Analysis Graph
Zenoss Code Analysis Graph
Zenoss Code Analysis Graph

Both of the graphs clearly show a plateau in the quantity of code committed to the respective community edition code repositories. There may be a number of explanations for the plateau, perhaps heavy re-factoring work clears the space required by new features. Though, realistically I doubt that re-factoring would be capable of continually reducing the size of the code base in order to make way for new code.

The plateau look suspiciously like evidence that open core software, at least in the network management world, tends towards a functional ceiling.

An exploration of open core licensing in network management

Open core refers to a business strategy employed by some commercial open source companies. The open core strategy is popular amongst companies within network management.

The open core strategy is largely defined by creating an open source community product that is freely given away, and another product, the enterprise edition, that is sold as a regular commercial software product.

The open core business model is useful to software vendors because it permits them to build a community surrounding the open product who will form the nucleus of the people who upgrade to the enterprise product.

The enterprise product is useful because it is packaged and sold in the same way as proprietary software. One of the major pluses for the open core strategy is that, having a paid for a product with all of the sales infrastructure that implies, fits in with many company’s purchasing processes. Tarus Balog, project lead of OpenNMS, posted about how his pure play open source business sometimes struggles with companies who expect to purchase software, rather than deploy the software for free and pay for training and implementation services.

Open core as the new shareware

The open core strategy has been likened to shareware, a software business model pioneered by Andrew Fluegelman, Jim “Button” Knopf, Bob Wallace et al in the late 1970s and long favoured by small Independent Software Vendors. Under the shareware model, the publisher distributes a limited version of the software that is either time limited or with key features disabled, in the hope that users will find the product useful enough to upgrade to the full version.

The shareware product is usually upgradeable to the full version by entering a product key supplied when the full version is purchased.

Whilst there is at least a grain of truth in the comparison there are some key differences between shareware and open core:

  • Key features are missing – open core software is useful in and of itself. An open core product that isn’t functional will not gain traction with a user community;
  • No community contributions – open core companies are keen to develop a community around their open core offering and hope/expect the community to contribute to the software eco-system surrounding the open core project;
  • Time limited – open core software is not time limited, you can use it for as long as you want.

The main similarity between open core and shareware business models is the desired end result on the part of the publisher. Both business models hope to up sell users to the full version of the product. The method used by both is also very similar, both business models withhold valuable functionality until the user upgrades.

Open core in network management

There has been quite a large influx of commercial open source companies into network management in the last few years, many with the largess of venture capital behind them. The most recent, RiverMuse, released the community edition of their event and fault management offering during 2009 and is following an open core strategy with the imminent release of their enterprise edition during early 2010.

In many ways network management is a perfect environment in which to exploit an open core strategy. Network management products are commonly structured around a central engine with add-ons integrating with third party networking hardware and servers.

The enterprise product is built around the core engine with a number of add-ons not provided in the community edition. The dual product strategy is most clearly taken by Zenoss who provide an open core engine but withhold many useful add-ons for important enterprise services like Microsoft Exchange and Active Directory. Whilst anybody could use the core engine to write their own add-on to provide the same functionality, many organisations find it more efficient to pay for a ready made and tested solution.

Vendor perspective

The pros and cons of the open core business model from the vendor’s perspective.

Open core licensing

The central part of an open core strategy is the dual licence. The community edition product is licensed under an open source licence, the enterprise product is usually licensed under a proprietary licence. Sometimes, when copyright or licensing issues intrude, the enterprise product also has an open source licence. Groundwork Monitor Enterprise Edition is a good example of an enterprise product having an open source licence. Dual licensing is only possible if you hold the copyright to all of the code, or have the agreement of the third party copyright holders to distribute under a restrictive licence. The same applies to any libraries distributed with the enterprise product.

If the enterprise product is licensed under an open source licence then there is always the danger that a customer may release the product in public, including the source code, meaning that potential customers no longer need to purchase the enterprise product in order to get hold of the value added features.

A fork in the road

A rival copy of an open source project based upon the same source code is called a fork.

As the core community product is freely available to anybody, there is a danger that a third party could create add-ons to the community edition and sell them in direct competition to the open core company. Whilst there is a danger of a competitor emerging to utilise the community product, there are some very good reasons why it won’t happen.

The competitor would be barred from using trademarks from the community edition in their product name or website. Consequently, it would be very difficult to promote the add-on to the desired audience. Trademark issues were one of the causes of the Icinga Nagios fork for instance.

In order to get around the trademark issue, the competitor would be forced to fork the community edition and release it under a new name. They could then sell an add-on product. Plainly the original community wouldn’t know anything about the fork and it would take a lot of marketing effort, in an already competitive market, for anybody to notice.

With the original community largely closed off, the competitor would have to start afresh and build a new community from scratch. Building a community takes time and money, external investment would be a very useful way to kick start the process. The competitor would not make a particularly attractive target for investment given that it doesn’t own any of the intellectual property of the fork.

In addition, the competitor would need to be absolutely certain that there are no source code or other artefacts which are being distributed as an exception to the community edition licence. There may also be clauses in the licence that have been inserted to guard against forking. The Zenoss licence contains just such a poisoning clause for instance.

Whilst forking is a danger to any commercial open core company, it does not appear to be a very pressing danger in practice.

Open core strategy

The open core strategy employed within network management companies encompasses quite a high degree of differences. There are companies like Hyperic who have pursued a pure open core strategy very successfully, controlling all of the software in both the core and enterprise products.

At the other end of the spectrum, Groundwork Open Source have executed more of an aggregation strategy, by bundling together well known open source projects together and making them into an enterprise network management platform with their own glue software.

The aggregation strategy could be considered to be more in keeping with the open source philosophy of software reuse. There are a number of advantages and disadvantages to the aggregation strategy. The main advantage being that by reusing a lot of best of breed components you get to market much faster than starting off from scratch. Though the vast array of licenses used by the various open source projects are likely to keep a good number of lawyers busy trying to sort out all of the requirements. Some open source licenses may well preclude use of the software within a commercial setting. Porting the software to new platforms is also likely to be difficult, you can only support the union of all of the platforms supported by the projects. Without the agreement of the project leads, you may have problems with trademark use, especially if you wish to market your software as being powered by the project in question. Many open source projects, Nagios being a very good example, do protect their names quite vigorously.

On the positive side, if you can leverage existing projects, you will have a number of communities ready and waiting to be up sold to your enterprise product.

Community perspective

An exploration of open core from the community perspective.

The Open Core Functionality Ceiling

One of the most difficult balancing acts for product managers of open core products is knowing which features should go into the community product and which should go into the enterprise product.

Does having an open core strategy mean that there are features that will never appear in the community product? Does the requirement to provide sufficient leverage to the sales VP provide an artificial ceiling for the functionality of the community product?

In a fully functioning open source eco system, the community would tend to close the gap between the community product and the enterprise product. Plainly an open core company is not going to be very comfortable with the value proposition of the enterprise product being undermined by the community.

Community contributions in an open core world

One of the problems with the open core strategy from the vendor perspective is that you need to be careful with how you handle community contributions. In the case where the company takes no third party contributions this isn’t going to be a problem, all of the engineers are on the company payroll.

If the company accepts third party contributions things become quite complex. In order to create a proprietary version of the software you either need to own the copyright to all of the software or have some kind of agreement to use the software in that way. A good example of such a third party contribution agreement is the Rivermuse contribution agreement.

The Rivermuse agreement must be signed each time a contribution is made. Whilst, from Rivermuse’s perspective, the agreement is absolutely necessary, I would think that the terms might make third party contributors think twice before agreeing to it.

Not only do Rivermuse have the right to sell your software without compensation, you have to assign the copyright of your work to Rivermuse. They also have the right to apply for patents based upon your work. If you submit your code, you could find yourself being sued for patent infringement by Rivermuse for discoveries that you made in the first place.

The OpenNMS Project Contributor Agreement, like the Rivermuse contribution agreement, also mandates that contributors assign copyright to the OpenNMS Group. The major difference is that the contributions are effectively owned by two parties, the contributor themselves and the OpenNMS Group, an arrangement known as dual ownership. The contributor also grants the OpenNMS Group a licence to use any patents contained within the contribution.

Open source etiquette

Many open source projects are written by people who gain no financial benefit from doing so. Open source software has been around for long enough that certain modes of behaviour have become the norm. One of the norms is the expectation that anybody wishing to incorporate an open source project into their own offering will ask the lead of that project for permission.

One of the dangers that commercial exploitation brings to the open source community is that the norms may be trampled upon. Is a company backed by outside investors likely to take the project owners views in mind when they have their own shareholders to concern themselves with. I’d like to be a fly on the wall in the board meeting when the VP of engineering explains to the board that a certain path cannot be followed because an open source project owner hasn’t agreed to their work being used, especially when the company would be perfectly within their legal rights to use it.

If a project lead sees their project being exploited by open source companies will it become a motivator to improve the software or will it become a disincentive?

Conclusion

Whilst there are many issues surrounding open core as a business strategy, it cannot be denied that an awful lot of high quality open source software has been written in pursuit of it.

When one surveys the open source network management landscape from before the open core invasion, it is hard to see how the user community has lost out.

Ipswitch acquires Dorian Software Creations Inc

Ipswitch, the people responsible for creating What’s Up Gold, have acquired Dorian Software Creations. Dorian Software are publishers of event log management software.

Dorian’s event log management solutions for Windows and Syslog environments include:

  • Event Archiver for automated collection, centralization and secure storage of log data;
  • Event Analyst for event examination, correlation and comprehensive reporting for audit and compliance;
  • Event Alarm for monitoring, alerting and notification on key defined events;
  • Event Rover for on-the-fly forensics and log data mining.

Dorian products are scheduled to be available from Ipswitch in March.

Open source network management buzz comparison 2009

I did a comparison of the buzz for the leading open source network management tools in 2008 so I thought it would be interesting to do the same comparison for 2009 and see what’s changed.

As I did last year, I’ve compared the number of searches for the project name using Google Trends. As always, this post is not intended to be indicative of the usefulness of a particular tool to your requirements.

Open Source Network Management System Trends

Firstly a comparison of the major players in open source network management: Zenoss, Hyperic, Nagios, MRTG and OpenNMS. The most striking thing about the graph to me is the decline in searches for Nagios. From the middle of 2009 things have been declining quite steeply. MRTG has been declining though it just looks like a continuation of the decline evident for the last few years.

Open Source Network Management System Trend 2009

A Comparison of the Nagios Ecosystem

Whilst the above graph showed a reduction in the relative number of searches for Nagios, perhaps the Nagios ecosystem graph can explain it. Icinga, a Nagios fork, was created during 2009 and may be responsible for at least some of the decline. Icinga appears on the graph during late April and has a steady presence throughout the rest of 2009 save for a small period during the Christmas break.

A Comparison of the Nagios Ecosystem 2009

Open vs Closed Network Management Systems

Given that 2009 was a year of recession in many countries, perhaps it won’t surprise too many to see so many of  both the commercial open source and proprietary tools trending downwards. I suspect that 2009 was a tough year for winkling money out of IT budgets.

Open vs Closed Network Management Systems 2009

Conclusion

All in all an interesting year. Apart from the Icinga/Nagios episode it seems odd that none of the tools has made  significant progress during 2009. If open source tools were to make a move against their proprietary cousins you would assume it would be 2009 given the economic background. Budgets have been tight, so why haven’t open source tools made progress in these recessionary times?

Network Computing rates PRTG #1

The German magazine Network Computing has done a comparison of four well known network monitoring packages: PRTG 7.2, What’s Up Gold 14, Solarwinds Orion Network Performance Manager 9.5 and ManageEngine OpManager 8.

From all these points of view we can only advise those who are looking for a good monitoring product to write PRTG Network Monitor right at the top of the list of products to look at. For Network Computing this product is, as previously, still the reference.

PRTG 7.2 came out on top!

RiverMuse has arrived

After a protracted wait, RiverMuse has finally released its open source fault management system. Binaries for Fedora Core 9 are available for immediate download. More technical details when the source code download link works.

Update: oops, bit early on this, RiverMuse isn’t officially released until 5pm today, 28 July 2009.

Rivermuse release iminent?

Looks like RiverMuse may just be coming close to a release next week after a considerable delay (around six months) if the latest RiverMuse tweet is authoritative.

RiverMuse is supposed to be a next-generation systems management tool that just happens to be open source as well. With the people involved in RiverMuse, including such network management luminaries like Philip Tee, Predrag Mutavdzic, and Mike Silvey, we can expect big things from them.

More news when the release actually happens…