Hub projects in open source network management

Almost as a doodle I thought I’d create a graph depicting the dependencies between a selection of open source network management projects.

Once I’d done it, it occurred to me how much just about everything depends on just a couple of projects or project variants of, RRDTool & Net-SNMP.

A Selection of Open Source Network Management Dependencies

The main conclusion I draw from the above graph is that if you wish to create a thriving platform for open source network management, you’d better have something like those two hub projects.

What is perhaps most surprising is not that there are hub projects, but that there are only two of them in the whole of network management. I think that there should be a lot more.

Creating open source software using Microsoft’s .NET framework

Whilst developing PowerTime, I’ve kept in mind the requirement that I need to ensure that the software can be built using only freely available tools.

Of course we will eventually provide an installer, but I like the idea that people can build the software on their own machine if they want to.

One way to make self building easier is to minimise the number of external dependencies required. To that end, everything is written in C# and only standard .NET framework libraries are used. No external libraries are required to create a functional build.

Build Environment

A free command line build tool is available with the .NET framework called MSBuild. The tool uses the same build files used by Visual Studio so duplicate maintenance of separate build scripts is not required.

Unit Testing

One area of .NET development positively overflowing with options is unit testing. A number of very good open source tools exist, including the excellent NUnit framework that I’ve employed.

Microsoft does have their own unit testing framework nicely integrated into the IDE, but it is only available to people who have stumped up for Visual Studio Professional 2008.

The only real problem with NUnit is that it isn’t integrated into Visual Studio. A solution is available through TestDriven.NET but unfortunately that is a (modestly priced) commercial product. Fortunately it doesn’t impact anybody else so I may buy it anyway because it would certainly improve my productivity.

A binary installer for NUnit is freely available so satisfying this code dependency shouldn’t be too onerous.

Development Environment

The following development environment options are available:

  1. Use the command line compilers supplied with the .NET software and use your favourite text editor; or
  2. Use the free Visual Studio Express IDE for whichever language you are interested in; or
  3. Use the (paid for) Visual Studio Professional; or
  4. Use the open source mono project on Linux with your favourite IDE.

Options 1 & 2 are not open source, but they are free. Option 3 is a paid for option and most certainly isn’t open source. Option 4 is completely open source and free. Only problem with mono is that it isn’t complete as yet. Mono has implemented most of .NET up to and including v2. For me that isn’t a problem because I’m targeting .NET v2 anyway. If you need features beyond v2 then you’d better look at the other options.

I’ve gone with option #3 because it is the most productive for me and I’ve already got the software. Fortunately using Visual Studio Professional doesn’t impact my goal of ensuring that anybody can build the code because if you haven’t got it you can use either MSBuild or Visual Studio Express.

Code Coverage Tools

One of the central tenets of unit testing is that you’ve got to make sure that you are testing as much of your code as possible. Believing that your code is being tested when it isn’t is far more dangerous than knowing it isn’t. That’s where code coverage analysis tools come in. Whilst a number of options exist (NCover & dotTRACE) for .NET developers, none are available for free.

Whilst JetBrains offers their IntelliJ IDE product for free to open source developers, I couldn’t see a similar offer for their .NET based products.

Again, whilst it would be nice to have a free high quality tool available to everybody there are no code dependencies to resolve with a code coverage tool. Those who have one can run it against the software without affecting those without it.

Conclusion

Nobody said that developing open source code on Windows was cheap. There are a wide variety of high quality tools available, unfortunately they cost money and most certainly aren’t open source.

Having said that, a viable development environment can be created either for free or cheaply. When compared, from an open source development perspective, to the Java environment, .NET doesn’t come out too favourably.

Perhaps Microsoft, if you’re listening, you could provide software like code coverage tools as a standard part of .NET just like you’ve already done with the compilers. Just very simple command line tools would be fine…no need to tread on the toes of the commercial ISVs.

Open source software from a VAR perspective

How can a network & systems management Value Added Reseller (VAR) benefit from offering open source solutions and what are the potential problems?

Any VAR not offering genuine added value is going to be left out in the cold. A “SKU” VAR, a company that expects to take orders with the minimum of fuss, are going to have a problem. But then, that’s the way commercial software like What’s Up Gold and Solar Winds Orion is going anyway. The internet is disintermediating the relationship from producer to customer pretty well everywhere especially in the close to zero friction world of software.

With open source software I suspect the effect of disintermediation will be felt most strongly. One of the most attractive aspects of open source is the community ecosystem. The last thing a potential open source adopter wants is to be cut off from the community. The direct relationship between user and producer is one of the main benefits of open source.

So how can a VAR get in on the action?

  • Create your own project by forking one or more existing projects. You effectively become the creator of the tool so you have the relationship with the user; OR
  • Morph into a consultant for existing projects. The only hope any VAR has of getting in on the act is through expertise by providing training and implementation help; AND/OR
  • Supply hardware appliances powered by open source tools.

Fork off

The nature of open source means that the source code is available to everybody. Why can’t a VAR create their own fork and create service offerings around it? There are two reasons why this approach is unlikely to succeed:

  • The project’s name is likely to be trademarked. So, when a fork is made it cannot be named after the original project. Consequently whatever community revolves around the original project isn’t going to follow you. You will therefore have the long and painful process of building your community from scratch. The desired revenues from consultancy services and training are likely to be long in coming;
  • Not too many VARs have the necessary skills or resources to successfully fork a project and continually keep the project up to date. Your developers are unlikely to be maintainers of the forked project so you will need to continually merge mainline project changes into your code base.

New wave nirvana?

The main problem with VARs morphing into open source consultants is that the project company may wish to keep all of that nice lucrative training and consultancy to themselves.

After all, most of the revenue garnered from “big 4” (HP Openview, IBM Tivoli, CA Unicenter, BMC Patrol) sales come from training and consultancy services not from license sales.

The new wave open source network & systems management companies are all venture funded. Somewhere in their respective business plans I’m sure training and consultancy feature very highly in their monetisation strategies. Indeed the new wave deliberately sacrifice some of the lucrative “big 4” software license sales in order to get hold of the training and consultancy business. I doubt they are going to be too enthusiastic cutting other people in on the deal.

Older is better?

On the other hand, there are a number of self funded projects like OpenNMS and Nagios. Do they offer better opportunities for VARs?

Nagios has a list of approved solution providers who purchase 3rd line support from Nagios Enterprises. The customer knows the solution provider is approved by the Nagios project and is likely to be locally based, the solution provider gets direct support from the project founders and the project itself is able to control the quality of the solution provider and gain income from support contracts from them.

Growing an organic support infrastructure is a clever move by Nagios. Setting up a worldwide support infrastructure is expensive. Nagios are able to leverage their website and trademarked brand in order to build an eco-system to deliver services and training worldwide.

Can’t sell the software…

If you can’t sell the software and you can’t provide training & implementation services what are you supposed to do? One solution is to sell hardware appliances based upon open source tools.

Zenoss has a good example of a hardware based network management solution just ripe to be sold through a VAR channel. Nagios has a whole host of recommended appliances too.

Hardware appliances hold the easiest route for VARs into the world of open source. In fact a hardware appliance based upon open source tools is no different than a proprietary appliance so far as the VAR is concerned.

Conclusion

Network & systems management is an area in which open source solutions have traditionally gained a lot of traction. Network management suits open source perfectly. The influx of venture capital into the existing open source ecosystem is only going to accelerate open source adoption.

So what’s a VAR to do? Open source is accelerating the death of resellers who fail to add value. So long as your people are able to add value to the customer then open source may well open up a huge opportunity for you. Instead of being tied to a single vendor’s offering you will be able to mix and match from a huge range of open source solutions.

Lessons learnt writing open source software

I don’t suppose many of you know my dirty little secret: I’m a failure. There I’ve said it. I founded an open source project and failed miserably. Sorry! Please wait there whilst I go outside to self-flagellate…that’s better the open source gods have been salved.

Back in 2005 I knew there was a tool missing from Apache for ad-hoc searches on access logs. Now, I mean a tool for performing ad-hoc searches not the programmable type. You can use AWK, a very flexible text manipulation tool, for that.

What I wanted was a tool for finding each entry in my Apache access log of all requests for a specific page or from a specific referrer. Something quick and easy without the need to learn complex commands. Quite often I find something in AWStats and I want to check it out in the raw log.

You’re probably thinking, why not just use grep? Well, grep doesn’t work very well in access logs. Say you want to find all accesses for a particular page. Greping will find all page accesses but it will also include all pages in which the page is the referrer (for images and the like referenced by the page) too. So, you will still have to sift through all of the other accesses to find what you’re looking for.

Logfilter treats each field separately so you can search on a specific field, or multiple fields.

Unfortunately, I suspect that a combination of the design goals and project scope seeded the project’s failure.

I may have been scratching my own itch, but how many others have the same one? The scratchiness of a problem is likely to be the main factor that makes a project succeed. If there is not commonality of need, then a community is unlikely to form to scratch the common itch.

The second problem comes in the form of the design goal. A tool that can be used quickly and easily without complex options necessarily limits where you can go with the tool. Every time I thought about adding a new feature I knew that the goal would be compromised because it wouldn’t have been simple to use any more.

In my experience, “open” development isn’t that much different from closed development when there’s just the one of you. Open development, when performed to an audience of one, is quite like closed development. Given two eyeballs, all bugs are deep to paraphrase Eric S Raymond. 🙂

Open source network management comparison 2007

Mark Hinkle from Zenoss sent me a link to an interesting document he prepared yesterday.

Open source network management download comparison 2007

I think Mark may be over egging Zenoss “clear market leadership” but without any doubt their growth over the last year has been impressive.

Perhaps the most impressive thing to take away is that all of the projects featured have grown over the last year.

Open source network management download comparison

One of the great things about sourceforge, apart from the cool services they provide free to open source projects, is that they provide statistics about the projects they host.

One of the stats that sourceforge provides is a history of project downloads. You can’t compare the stats though. So I thought it would be interesting to compare the downloads for the major open source network management projects.

The volume of downloads is indicative, like search trends, of the relative mind share for each project. Download volume isn’t a perfect measure, but it is one of the best available. I doubt even the projects themselves have an absolutely accurate idea of how many installations they have.

I entered the download data from sourceforge for the last year into a Google Spreadsheet. I then graphed the data. See the graphs below. The data covers the previous year, from November 2006 through to October 2007.


Figure 1: Open Source Network Management Projects by Monthly Downloads


Figure 2: Open Source Network Management Projects Total Downloads

I’ve compared five projects: Nagios, OpenNMS, Zenoss, Hyperic & Groundwork Open Source all of whom host their downloads at sourceforge.

I doubt the graph will surprise too many people. The graph is similar to the Google Trends data. Both Nagios and Zenoss are vying for the top position. What has surprised me over the last year has been the stability of the number of Nagios downloads.

If the growth of the “new wave” is coming from other open source projects, it isn’t coming from either Nagios or OpenNMS, the most mature “old skool” open source network management projects.

I don’t think there is much doubt that both Zenoss & Hyperic have brought commercial levels of setup and configuration to the open source network management market. And yet, Nagios a tool that relies on manual configuration, is still gaining traction.

Once you have a loyal community, by delivering and supporting successive releases over an extended time, users are loath to move to another project.

Perhaps perversely, the harder a tool is to learn, the more reluctant users are to migrate to another tool. A kind of open source Stockholm syndrome.

I’m sure that all of the new wave players understand the value of community…that’s why they are going hell for leather building them. 🙂