OpenNMS 1.3.3 Released

Once again it is with great pleasure that I get to announce the availability of an new OpenNMS, version 1.3.3, the next step down the long path to OpenNMS 2.0.

Most of the packages should be uploaded to Sourceforge by now, although we are still working on Solaris, Debian and Ubuntu versions. We hope to have those out in the next 48 hours or so.

One of the problems with releasing OpenNMS is that despite the fact that we are mainly written in Java, there is some C code required for OpenNMS to be able to send pings, create graphs (with RRDtool) and use the “iplike” function in PostgreSQL. While these don’t change very much, due to the differences in each distro we have to build distro-specific packages for OpenNMS.

We hope to change that, starting with this release, and the “iplike” function has been moved to its own package.

If you are upgrading OpenNMS, there is no need to install this package since the “iplike” function should already exist. With new installations, however, it is strongly recommended that you install this package. If you are building from source, please check out the iplike source tarball and follow the instructions there. Once it is installed, you do not have to install it again, as the OpenNMS installer will check for its existence. If it doesn’t exist, OpenNMS will install a PL/pgSQL version of the function that will work, although not with the same level of performance as the C-based code.

Also, we have changed the default storage method for data collection. We used to store “as polled” (5 minute) data for two months, and MIN, MAX and AVERAGE data per hour for a year. This created very large RRD files and often stressed the disk I/O of the server. For comparison, MRTG stores as polled data for one day.

So now by default we store as polled data for a week, hourly data for two months, and MIN, MAX, AVERAGE data per day for a year. Feel free to change this to fit your needs, and if upgrading be careful with the .xml file merge.

This is a pretty exciting release with a number of new features.

The new features include Trouble Ticketing API that we hope will allow OpenNMS to integrate with a number of other applications. The first plugin we created was for CentricCRM (also a Java application), and we plan to add others in the future.

We also have a Dashboard user interface built using the Google Web Toolkit. Web 2.0 here we come. (grin)

It was over a year from 1.3.1 to 1.3.2, and five months to 1.3.3. We are hoping to get 1.3.4 out much more quickly. We will continue to document and improve on what we have in 1.3 already, and add features such as an XML data exporter and Smokeping integration. The next bit of C code will be to make RRDtool optional (although we plan to always support it) with JRobin being the default (as it is now).

We are also expecting great things from the third annual OpenNMS developer’s conference: Dev-Jam.

At the moment we have people from four countries attending and there are still some spots left (although the price discount is no longer in affect). We also have sponsorship opportunities available (those never run out).

From all of us at the OpenNMS Project, I hope you enjoy using this as much as we enjoyed making it.

Services, Part 3: The Future

The Future

Occasionally OpenNMS gets mentioned alongside well funded “open source” management companies. Since we haven’t announced any multi-million dollar investment, the press often ignores us. But I much prefer it when we get mentioned favorably in the company of HP and IBM. That’s where I want to see the community take OpenNMS.

So, how do we get there?

First off, it takes time. You can try, but you can’t “buy” a community. We’ve worked very hard to build the OpenNMS community over the last seven plus years. We’ve done it by focusing on two things: OpenNMS will always be free, and OpenNMS will never suck. By trying to keep the .org and .com sides separate, we hope that those people active in the community will never fear that we would take their work and commercialize it. I like saying that you can’t stick nine pregnant women in a room for a month and get a baby. Some things take as long as they take, and no amount of money can speed that up.

It has taken OpenNMS over seven years to get where it is today, and I believe that some investment at this time would go a long way toward making the product stronger. Not $24 million, $10 million or even $4 million, but something more along the lines of $2 million.

Why “just” $2 million? Two main reasons: I don’t think the market is ready to support a large open source management company, and OpenNMS is not ready to spend more than that.

Since I like to explain things, let me ‘splain.

When a company takes in venture capital, the VCs don’t want it to sit around in the bank. They have banks, nice banks, that could take better care of their money than most start-ups. They want it spent, and they want a return of 5 to 10 times their investment in about 5 years. Thus a company that raises $20 million has only a couple of years to become worth well over $100 million dollars. That’s a mighty big challenge, and my guess is the temptation to raise that money on the backs of their customers will become really strong. How would they do that if it is open source? If it was truly open source, they couldn’t, but by using a software subscription licensing model they can lock the customer in, just like HP tries to do with OpenView.

I really don’t think any of the open source options out there, including OpenNMS, can both be true to open source and become worth that much in a couple of years. It is a totally new concept for many enterprises to not be forced to pay money for software. Current commercial OpenNMS clients are obviously ahead of the curve (they are quite the smart bunch, and good-looking, too). Many companies don’t believe that they can get something equivalent to much of what OpenView and Tivoli provide for considerably less money, or that software developed by a consortium can work better than one developed in a monolithic organization. It will take time, I think a couple of years, before it will be possible to really bring something like OpenNMS to the mainstream. Everyone is looking for the “catch”, the “gotcha” and the prevalence of “open source” as a marketing term isn’t helping.

Even if OpenNMS was able to raise many millions of dollars, I don’t think what we’ve started can be rushed, and thus I don’t think i could spend it wisely. I am so blessed to be able to work with the folks I work with in the OpenNMS community that I won’t do anything to jeopardize it, even if it means short term financial benefit for myself.

So what would I do with $2 million? I call it my $2M/2 year plan. Over the next two years I would use that money to focus on the real intellectual property of the OpenNMS Group: our knowledge of network management. We make a little money each year, and thus we have a good idea of what it takes to run a services business. OpenNMS could definitely use some polish, in terms of documentation and ease of use/installation, that an infusion of cash would enable. Plus the biggest comment I get when talking about OpenNMS is that few people have heard of us. I’d like to be able to market ourselves a little better. It’s a great testament to the product that we’ve been able to survive solely on word of mouth advertising.

Now, the biggest criticism of a services company is that you have to find and hire an army of people to provide those services. Aye, there’s the rub, eh? At OpenNMS we will get around that in a number of ways.

The last two people I’ve hired have been taken directly from the Order of the Green Polo (OGP). The OGP is the core team behind OpenNMS. Imagine hiring someone and having them be productive days before they even start their job. The people in the OGP are in it because they love OpenNMS, and you can’t find better. I’d like to hire them all.

So what happens when they run out? The “top secret” success of the OpenNMS Group is that we’ve learned how to use a core group of dedicated people to inspire others, and we have formalized a process where you can take a motivated person and quickly turn them into someone who can help others with the product and maintain a “laser-like” focus on the customer. They might not know everything in the beginning, but they will be helpful and, better still, they will know where to go for the “everything”.

Identifying and training people like that is hard, but we’ve learned both through experience and through services clients of ours like Rackspace that it is possible.

One final criticism of services companies is that the people are always on the road. The answer to that is simple: we won’t be doing the traveling. We are working though the community to find partners around the country and around the world to deliver on services related to OpenNMS. The OpenNMS Group will focus on support (which can scale), custom development (which drives new features) and training, while our partners, who already have great relationships with their own clients, can have the services revenue associated with deploying and customizing OpenNMS. It’s worked well in Italy, and we have plans underway for partners in the UK, the rest of Europe, Turkey and Saudi Arabia. Since we don’t want to “own it all” we can make this happen, and use certification to insure a high level of quality.

I think getting all of the kinks out of this will take 2 years. Even if I had more money, I don’t think it can be done any faster. But I am certain by the end of those 2 years OpenNMS will be poised to become a real contender for replacing OpenView, and at that time we will have a story for the VCs that will make sense.

Now the only trick is to find an angel with $2 million to invest willing to wait 2 years. Answers on a postcard please …

It’s now about 3 hours since I started writing this, although with about a hundred edits, so I just want to sum up and go to bed. The future of software is in services, be it direct “I pay a person to do stuff for me” or software “as a service”, like Salesforce.com. Services are key to the success of open source projects. If it weren’t for services we’d have never discovered the market need for such things as an integration between CentricCRM and OpenNMS, much less gotten it done in less than a month.

Hybrid business models such as the “shareware” model or the “OpenView” model are short lived at best. Being totally free, products like OpenNMS will own the bottom, constantly driving software licensing revenue down and without a strong service model there is no way to make up for that loss. Finally, those companies that focus on service as the product, and can successfully scale it while maintaining a high quality of customer experience, will ultimately prevail.

I’ve bet the farm on it.

Services, Part 2: The Present

The Present

I’ve talked in the past about a number of “open source” business models. There’s the MySQL “dual license” model, where all the software is available under an open license but a proprietary license can be purchased. There’s what I call the “shareware” model where some software is offered under an open license and the rest is proprietary, and then there’s the “parasite” model where companies try to build a business by exploiting the work of other open source projects. I am sure there are many more, and I’d like to add one called the “OpenView” model. This is where a company wants to use the term “open source” to create a community like the OVForum, but keep a portion of the software closed so they can reap licensing revenue.

The current darlings of open source seem to be those who have managed to sell investors on an open source business model based on license revenue. They scoff at services businesses. One recently stated that in order to be successful, an open source company couldn’t have more than (approximately) 25% of its revenue from services.

So, what’s wrong with the statement that an open source company cannot be successful unless 25% (or less) of their business is services? Well, if I’m doing the math right, that means 75% (or more) of their revenue must come from something other than services, and that can only be software licenses.

The power of open source ultimately comes from the community. The community will work together to build something wonderful, and almost by definition for that to be successful the community must own the final product. If the product is actually owned by a specific entity, for the purposes of revenue generation, the potential of that community is lessened, and so is the product.

Is there anything wrong with that? It depends. The OpenView community was one such community and it thrived for a time, but ultimately it faded. Certain “open source” management companies seem to have the goal of replicating OpenView (hence my label of the “OpenView model”) by creating software that is easy to expand via plugins (although the core code or enterprise-level code is closed), but since people are fleeing OpenView in droves I don’t believe that a similar model is sustainable for a long time.

If you write plugins for these companies, please don’t take what I am saying as an insult. As an open source guy I’m a firm believer in that what works, works. Almost all of these firms provide a lot of code under the GPL and that makes them head and shoulders above OpenView, but I have to wonder about a company that bases the majority of their revenue of software licensing and also calling themselves “open source”. What would happen if someone in the community contributed a patch that threatened that revenue stream? How can one encourage code contribution when the more code that gets contributed results in less money for the company? They can try to funnel the effort into “plugins” and tell everyone else to ignore the code behind the curtain, but that really defeats the purpose of open source.

These firms may be able to pull off creating a company worth a lot of money just by providing a useful, albeit non-free, product, but as free products become better and better, their software subscription model becomes weaker. I’ve heard it called “owning the bottom” and after originally disagreeing with the concept, I’ve become a believer. I’m betting that the next OpenView will come not from another commercial software company, but from an open source community, and I’m also betting that community is OpenNMS.

In the long run it will be the company that does services best that is worth the most. Even now the trend is moving toward services or software as a service. Which brings me to the future: what makes a services company successful?

Services, Part 1: The Past

I’ve been doing work in “open source” for over six years now, and I spend way to much time thinking about it. In fact I started writing this post when I couldn’t sleep after midnight one night, and I decided to get out of bed and write down my thoughts. As Morpheus says in the Matrix movie, there is a difference between knowing the path and walking the path. Having walked on this path for so long, I’ve come up with some ideas about how open source will impact the future, if not the present, and I wanted to share them (if just to formalize them for myself).

While I donate a large amount of time to The OpenNMS Project, my salary (and thus my mortgage) is paid by The OpenNMS Group. The OpenNMS Group is a services company. We don’t sell software, but we sell services in the form of support, consulting, training and custom development. For some reason, being a pure services company is seen by many who really ought to know (i.e investors) as being a “lifestyle” company, one that “can’t scale” and thus isn’t really worth investing in. There were times when I used to think they were right, but as with anything as new and novel as building a business on open source is, I’m now prone to believe that it is more of a case in that they can’t believe that the days of selling software are ending. Something, namely services, must rise to take its place.

The OpenNMS Group has been operating over the last several years with a strange business plan. We call it “spend less than you earn”. I know it sounds crazy, but it has worked for us so far (grin). But the time has come to seek out investment, and so I’ve been thinking of how can I explain what I’ve learned about open source to those busy investment people with necessarily short attention spans without requiring them to spend six years in the trenches like I have. The results are three blog posts: the Past, the Present and the Future. I plan to post these over the next three days, and I welcome any feedback my few but loyal readers may have.

The Past

The first real network management platform I was exposed to was SunNet Manager. It was one of the first graphical applications that allowed you to create a database of your network devices, and to both perform some tests (in the form of pings) on them as well as receive events (via either SNMP traps or log messages) from them. Sun developed it more as a toy than a serious application, and it wasn’t long before it was eclipsed by Hewlett-Packard’s OpenView Network Node Manager (NNM).

I got involved with NNM around version 2.2. HP had improved slightly on SunNet Manager but they had gone to great lengths to make it easy to integrate into their platform. The “open” didn’t stand for “open source”, but it was real easy to add new applications that would integrate via menus, graphics, and events. The actual NNM application did not do much – it could discover the network, place that information in a database, and receive events (as well as send “pings” and SNMP requests). Man was it ever fun to play with, and quite powerful despite its simplicity.

For example, suppose you were Bay Networks and you wanted to create a management application for your Wellfleet routers and Synoptics switches. NNM could discover those devices on the network, and using SNMP it would collect the sysObjectID so that it would “know” what type of device it had discovered. Using that ID, Bay could create custom icons that would display the device type, as well as menu items that would then launch external programs and pass to it information such as IP address. Then those applications could send events back in to NNM for display. Bay could focus on the management specifics for their equipment and let NNM handle all of the basic stuff.

This flexibility caused a huge community to spring up around NNM and then around the rest of the OpenView suite. HP provided an API that allowed people to write applications that tightly integrated with their platform, and a users group called the OVForum was started (although it looks like it is called “Vivit” now for some reason), without prompting from HP, to help bring the community together. I was deeply involved with the OVForum, namely through one of its founders, Paul Edmunds, who worked rather close to me in Charlotte, NC. Another person, Chip Sutton, who lived even closer in Durham, took the HP API and created one in Perl. Perl being the language of sysadmins at the time, it caused even more functionality to be added to OpenView.

At its peak around NNM version 5, the OVForum and its yearly conference boasted thousands of members, and it was hard to think of deploying a network management solution without it.

It has been my stated goal to replace OpenView with OpenNMS as the de facto management platform, so one may ask if OpenView was so great, why get rid of it?

Although OpenView was “open”, it was not “open source”. We were often frustrated as consultants working with the product with many of its shortcomings. Then, as HP grew and started to focus more on PCs, OpenView languished. Because the core of the system was closed, it could be hard to get it to do exactly what you needed.

OpenView VARs were often told they could expect 7 to 8 dollars in services for every dollar of software sold. While NNM was priced either for unlimited nodes or for nodes in groups of 250, some of the other OpenView components started using per node pricing which made the product expensive. Combined with services, IT departments started to see their budgets dwindle in order to implement OpenView.

OpenNMS was started to provide all of the functionality of OpenView (and other products) with none of the licensing costs. Being truly open source, all parts of it can be improved and changed where needed, thus it is very flexible and that flexibility reduces deployment time. Lack of licensing also reduces yearly support costs, which have been known to approach 25% of the purchase price of the original software. But the key here was to create a platform, open to all, but driven by services.

Exciting Travel Ahead: OSCON, Dev-Jam, LinuxWorld Expo

Having just returned from three weeks on the road, I’m not looking forward to being gone that long again, but some real exciting events are coming up for late July and August.

For the first time I’ve been invited to give a talk at OSCON on OpenNMS Case Studies the week of July 23rd in Portland, Oregon. I really like Portland and this will be my first trip to OSCON. I have to like people who have their headquarters in Sebastopol, California, an extremely beautiful place to live (I used to live outside of Santa Rosa). It seems to be a slightly different crowd than LinuxWorld, and I’m really hoping to meet in person some of the folks I’ve only known through e-mail in the past.

The week after that is one I look forward to every year: Dev-Jam. This is the one week of the year where many of the core contributors and soon to be contributors can get together and focus on our goal of making OpenNMS the world’s de facto management platform. We are getting ready to release version 1.3.3 of the code and I plan to have 1.3.4 tagged shortly after Dev-Jam ends. We have people coming from the UK, Italy, Venezuela, Belgium and the USA so far, so it should be a lot of fun.

Note: Friday is the last day to register at the reduced rate. If you are interested in learning about OpenNMS code and becoming part of the development community, Dev-Jam is the event to attend.

Another Note: Sponsorship opportunities still exist, too. Currently Google is the only confirmed sponsor, but we hope others will join them. Much like the Debian conference does with certain contributors, we underwrite the cost of Dev-Jam for members of the OGP as well as subsidizing the cost of the week for all attendees.

Finally, the week after Dev-Jam we are returning to the LinuxWorld .org Pavilion after a two year absence. We’re in great company this year, and I really love the .org Pavilion part of the show. Please track us down in the “Geek Ghetto” to say “hi”.

To Ethan, with Love

Whenever we go to conferences to talk about OpenNMS, invariably we get compared to Nagios. Often the tone is quite challenging, as in “How are you better than Nagios?” in much the same way that geeks are wont to argue “vi vs. emacs”, “KDE vs. Gnome” etc. and the idea seems that OpenNMS has to be “against” Nagios, which couldn’t be further from the truth.

Tarus in a Nagios Shirt

In my mind open source is about stuff that works. As a hack I love vi, but many of my developer friends are fond of emacs (or were before Eclipse came along). If I stuck an emacs guy on vi he’d be less productive, the same as if I were put on emacs. Does this make one better than the other? I don’t think so.

Ethan Galstad registered NetSaint, the precursor to Nagios, on Sourceforge two months before OpenNMS, back in January of 2000. So while Nagios is better known by far (10 times the number of Google hits on average) we’ve been around for about the same time.

The extra notoriety has actually been somewhat of a blessing, since some of those companies attempting to market themselves as “open source” decided to infringe on Ethan’s Nagios trademark versus OpenNMS.

While I’ve never met Ethan in person (although he is supposed to stop by Dev-Jam this year), we have been looking out for each other’s projects for several years now and correspond via e-mail a couple of times a year. Before we announced our NRPE and NSClient support, I made sure to let Ethan know we were going to do it and sent him an OpenNMS shirt. A couple of weeks ago he returned the favor and I now have a cool Nagios shirt.

His tagline “Only the NSA Monitors More” is priceless. I seriously wish I had come up with it, although it is only true in the context of Nagios. Ethan is also a really nice guy, and it’s sad that people try to commercialize his work and the work of the Nagios community. At least he can still keep his sense of humour about it all.

At OpenNMS we strive to make sure that OpenNMS will always be free, and that OpenNMS will never suck. Nagios helps with the latter by driving us to be a better tool. We may sometimes compete, but we aren’t enemies, and I’ll proudly wear my Nagios shirt.

But I must say that at OpenNMS we have really cool clients, and we recently got a new one in Hawaii. So let me point out that the picture above was taken on the black sand beach of the Waipio Valley on the Big Island of Hawaii. Score one for OpenNMS (grin).

Waipio Valley, Hawaii