I'm Too Sexy for my Shirt

Sometimes I read back over my posts and I am certain that I come across as an arrogant a$$hole. I really don’t mean to; I just can’t help but to speak my mind on things such as the meaning of “open source”, running a business, beer, scotch, whether or not the Patriots should be the first team in the NFL with an undefeated season and other important issues.

My one true talent is the ability to surround myself with people smarter than me. Their only fault is, perhaps, hanging with me in the first place. I can’t help but think that anyone outside looking in would think “man, that’s a great team, but what’s with the loudmouth?” (grin)

In my zeal in promoting OpenNMS and our free and open ways it might seem as that I hate everyone and everything not OpenNMS. That’s not true. I’m pretty much for anything that works, even commercial solutions, and if something else works for you, great. As a services company it would actually cost me to try to support our product in the wrong environment.

Believe it or not I actually like and at times get along with other people in the field of network management. Take Eric Dahl at Zenoss, for example. We’ve met a couple of times and he’s a nice guy (not quite as nice as Ethan Galstad, but who is). At barcampESM he promised to send me a shirt and I promised to wear it.

Is it true that black is slimming? Or does this shirt make me look fat?

On second thought, don’t answer that.

Anyway – a big shout out to Eric for the polo. I’m putting an OpenNMS polo in the mail for him tomorrow. Peace out.

Bugfest for 1.3.10 on Saturday, 2 February 2008

[The following is a note I sent to the lists today]

Gang:

I’ve been very happy to see a lot of activity on the lists and in bugzilla recently. OpenNMS is all about the community, and nothing pleases me more than to see lots of new people with a sense of ownership toward the project.

With that in mind, we’ve decided to have a “bugfest” to clean up as many bugs as possible for the 1.3.10 release. Those of you watching the subversion commit notices have seen quite a number of them fixed over the last couple of weeks, and we figured that if we all got together for a day we could knock the rest of them out.

Here’s how this works:

1) We will be coordinating this via the #opennms IRC channel on freenode. For those of you who don’t usually use IRC or never have, it is just a way to create a chat room where a group of people can talk. The IRC wikipedia article is a good place to start, and there is even a decent comparison of IRC clients.

When you get your software installed, you will have to connect to a server. In our case it will be “irc.freenode.net”. Then you will want to join the “#opennms” channel (the # sign is important, so don’t leave it off). Most clients with a command line use the “/” symbol to indicate an action, so “/join #opennms” will most likely get you to the right place.

You may also be asked to choose a nickname (or “nick”). Mine is Sortova. The channel is always active, so feel free to test it out before the 2nd.

2) Everyone is welcome, even if you don’t write code. We will be generating a number of snapshots that day, and help with testing (both the installation and functionality) is just as important as code changes. This would also be a good time to brush up on your wiki syntax skills as I’m sure we could always use some help with the documentation.

3) The focus will be on the 1.3.10 tracking bug, although any bug is fair game. The goal is to release 1.3.10 the week of the 4th.

I look forward to seeing some old friends and some new faces. I’ll probably start around 8am to 9am Eastern US time, but there will be people on the channel all day. No one has to stay the whole time, of course, so if you can only spend an hour or two that’s cool.

KDE on Mac OS X

This isn’t OpenNMS related, but Ben Reed, one of our team members, was just put up on Slashdot for his work on the KDE on OS X port.

w00t.

[Update: Every wonder was being on Slashdot looks like to the network? This is a traffic graph from a server providing KDE torrents]

Why All the Hate?

I got a note from Javier this morning with the title “Why All the Hate?”. It made me sit up, because I like Javier, a lot, and if my post on VC money came across as hateful, I want to try to patch things up.

I met Javier at last year’s SCaLE conference and took an immediate liking to him. He loves his company, his product and his people as much as I love mine. We’re hermanos. Heck, you can even rearrange the letters of his name to spell “i.e. Tarros Veloj”, which means we have almost the same name, too.

This is why I hate writing about the industry. For people who do not know me it seems like I hate VCs, I hate the big four, I hate the little four, and I hate their communities.

Let me quickly address those points in reverse order.

First of all, to the users of all those other network management products out there: if it works for you, great. Don’t even consider OpenNMS. There is no reason to replace something that works. If this means you can take a shareware open source product and make it do what you need it to do, within your budget (of both time and money), then it is a great solution. If it requires a fully commercial product to do what you need done, great. Sometimes it makes better sense to spend a little money to save a lot of time.

As for the “little four”, I don’t agree with shareware open source models and nothing is ever going to change that. That doesn’t mean I “hate” them. In the context of traditional software companies it makes a lot of sense. Heck, the creators of MySQL and Xensource can thumb their noses at me from their Ferraris. But I truly believe that years from now the projects that are still recognizable as open source won’t have a commercial software component. I’m betting my Ferrari on that.

The “big four”? Heck, they don’t know we exist. They are so big and so far away I’d be surprised if they cared very much about open source at all. The guys in the trenches tasked with making those solutions work are still cool, as I rediscovered at barcampESM, hangin’ with the guys from BMC and Tivoli. No hate, I just believe there will eventually be a better way.

And finally, VCs. I don’t hate VCs. They are out to make money just like I am. Shareware open source models have made a lot of money for some over the past few months.

I will admit that VCs scare me a little.

They scare me because, as I mentioned in a comment on Jack Hughes’s blog, the goals of a VC firm that invests in open source based on a software revenue model are not the goals of the community at large.

Case in point: if I used the free version of a shareware open source product, wouldn’t I benefit from having the entire software available at no cost? Wouldn’t I be able to contribute more if I could focus on new features instead of trying to replicate existing “enterprise” ones? Thus the community goal of getting as much functionality as possible into the open source part of the product negatively and directly affects software revenue.

I would love for someone to point out to me the flaw in my logic here.

This doesn’t mean I wouldn’t consider investment from a VC firm. If the focus is on revenues outside of software, then a strong community means a stronger product and ultimately more money. Thus VC investment works to drive the community and everyone benefits. I’ve even laid out how I’d do it should we decide to accept such an investment.

Note: for those of you who think I’m being naive and that I just don’t want to give up control of the OpenNMS Group, I already have.

So, enough of this for now. Time to get focused on 1.3.10. And remember, I’m not a hater, yo.

Sometimes OpenNMS Amazes Me

When I became the OpenNMS maintainer in 2002, I wasn’t sure if I was up to the task of not only keeping the project alive but making it better. I knew the trick would be to surround myself with people smarter than me, and I somehow managed to do that with the creation of the OGP.

In 2003 we had a client who wanted to integrate a monitoring system with their in-house customer management application. Due to the architecture of OpenNMS, it was pretty easy to set up an extremely scalable monitoring platform. The only problem was that there wasn’t an easy way to get the monitoring information into OpenNMS and then back to their system.

This was well before the Importer was created, so since OpenNMS was able to receive arbitrary events from external systems they contracted with us to write some code to make the integration easier. This resulted in the creation of the xmlrpcd daemon. It is a daemon that isn’t used much, if at all, outside of this one client.

Now, when a client contracts with us to write custom code, we strongly suggest that the modifications be added to the code base. The project’s main architect, Matt Brozowski, introduced the concepts of extreme programming into OpenNMS, which include things like unit tests. This means that as OpenNMS grows, even older, legacy code will be kept up to date as the unit tests will fail if changes are made to the code that breaks their functionality.

That system we wrote in 2003 has been in continuous production since then, monitoring tens of thousands of servers in four data centers around the world. It is running on OpenNMS 1.1.1, on Debian Woody and PostgreSQL 7.2.

Today we configured the migration servers running 1.3.9 on RHEL 5 and PostgreSQL 8.1. After working out some issues with SSL certificates in the test environment, it turns out that the xmlrpcd code works just fine, something like 20 releases, and 5 years, later.

That’s the kind of stability that rates the name “enterprise-grade”, and why we use it in our tagline.

Show Me Da Money (a Cautionary Tale)

Okay, I usually don’t like writing about the new batch of so-called “open source” management companies. Since we are trying to position OpenNMS against products like Tivoli and OpenView, they really aren’t who we are trying to compete against, and since we don’t have millions of dollars in investment it could come across as if I was a big whiner, boo-hooing about our poverty.

That really isn’t the case. We made a decent amount of money last year (made as in “net profit“, not “gross sales“), and I’d be more than willing to welcome in an investor or two who shared our vision. It’s finding such an investor that has been hard.

So here is a post about the “little four” management companies: Groundwork, Hyperic, Qlusters and Zenoss.

Let’s drop Qlusters right off the bat, since they aren’t really in the same “monitoring/fault” space as the other three. It just goes to show you that the guy who came up with the name really wasn’t doing his homework (grin). All three have raised quite a bit of money:

Groundwork: US$25 million in three rounds
Hyperic: US$10 million in two rounds
Zenoss: US$15 million in two rounds, including US$11 million announced today

This money was raised from venture capital (VC) firms. Now VC firms exist for one reason, to make money. The way they make money is with a “liquidity event,” which is when something happens to turn the assets of the company in which they invest into cash. This can happen as an IPO or an acquisition. Open source IPOs are rare, and with MySQL being acquired by Sun, the closest hope for an open source IPO in the near future is gone. So that leaves acquisition.

VCs want these companies to spend money, and quickly. They need to get the value of the company up to the point where their share is approximately ten times the amount they invest, and they tend to want it within 5 years. Examining our “little three” at a minimum they need:

Groundwork: US$250 million
Hyperic: US$100 million
Zenoss: US$150 million

That’s a lot of money, especially for a company that plans to give their software code away.

But wait, these companies don’t give their software away. Only a small part of it is open. A large portion of the revenue that is to drive these massive valuations must come from software licenses. By accepting this money they have pretty much promised not to open up the rest of their code. If they did, their software license revenue model would be in jeopardy, which would make the VCs very unhappy.

You don’t want to make the VCs mad.

Example 1: Last year I learned about Medsphere. Started by Scott and Steve Shreeve, their goal was to create better medical software using the open source development model. They accepted some VC funding, developed some code, and released it on Sourceforge. They were promptly sued by their board to the tune of US$50 million for releasing trade secrets. I’m serious – an open source company sued its founders because they … opened their source. And I’m not talking little bitty counts either – they were sued under RICO (Superior Court of California, Orange County, Case 06CC07475). RICO was designed to catch mobsters, but the severe penalties imposed by the law (i.e. they can seize your house, the kid’s college fund, your 401(k)) are used by unscrupulous lawyers to scare the bejeezus out of you. The case was settled last October, but the Shreeves are no longer associated with the project or Medsphere.

Example 2: One of the first companies to jump on the open source management bandwagon was Groundwork. Riding on the popularity of Nagios, they built proprietary software tools around it. President and CEO Ranga Rangachari made the rounds of all the industry rags with his vision for how to make money with open source. However, if you look on the Groundwork site today, his name has been quietly removed from both the Management Team and Board pages.

Remember, don’t make the VCs mad.

I have to wonder. If acquisition is the main exit for these companies, why would someone like IBM or Sun want to pay US$100 million or more when they could pick up, I dunno, something like OpenNMS for half that? (grin)

I sincerely want to wish Zenoss luck with their new investment. I enjoy the competition even if I dislike what I see as the shareware open source model, and it makes me even more determined to keep OpenNMS 100% free and open. But I don’t dislike the people at Zenoss (heck, one of my old friends is a sales guy for them) and I am very fond of the folks over at Hyperic as well (to the point of feeling bad about my spat with Doug MacEachern on Slashdot so long ago – he’s an amazingly nice guy).

But as a profitable project with a pure open source business model, we’re gonna be dogging their steps every bit of the way, providing for free what they provide for a fee, and diluting their revenue streams. At least I don’t have any VCs to get mad at me.

OpenNMS Austin Brunch

Since we at OpenNMS don’t put any “phone home” code in our software, it is sometimes hard to tell who uses it and even if they like it. So when I travel I like to meet up with any users in the area (OpenNMS users are always incredibly intelligent, stunningly attractive and possessing a rapier-like wit). With both archon~ and derrick from the IRC channel in the Austin area, as well as Coté, I decided to try and set up a brunch in Austin before I had to head for San Antonio.

The first goal was to determine a place. It was up in the air as late as yesterday afternoon, when a chance meeting provided both a suggestion and an interesting conversation.

When I was on the barcampESM monitoring panel, I noticed that a woman had arrived and was busily snapping pictures. This was odd enough, since, to begin with, there were more cameras there than at a Britney Spears court appearance, and the fact that she was, well, a she, made her stand out. She left before I could find out who she was (and I was curious as to where those pictures were going to end up) but later when I was drinking the new Shiner Black Lager (yum), she returned and I was able to introduce myself.

Her name is Michelle Greer, and she’s involved in Geek Austin which is how she knows whurley and why she was there taking pictures. Geek Austin seems to be similar to Triangle InterNetWorkers in my area. She seems to know Austin pretty well, and she recommended Z Tejas as a nice place for lunch. It was a couple of blocks west of where the barcamp was held, so based on her recommendation and the fact that I could find it I decided to hold the OpenNMS Austin Brunch there.

Frank Sheiness (~archon from the #opennms IRC channel) and Chris Bowman were in, and I invited Coté as well, but I had the feeling that he had probably reached his limit of “Tarus Time” for the weekend. Derrick from IRC was going to show up, but he had some delayed and cancelled flights coming back from Florida and didn’t make it home until well after midnight.


Me, Chris and Frank at Z Tejas

I got to learn a lot more about Frank and Chris’s company, Korcett. They provide broadband services to apartment complexes, and they chose OpenNMS as their management tool.

I asked them how many devices they were monitoring, and the answer was less than 50. This surprised me, since the learning curve for OpenNMS is rather steep, and usually people with smaller networks don’t have or take the time to learn it. So I had to ask “Why OpenNMS?”.

There was the usual reason of scalability. The network is small now but they hope it will be growing rapidly, and it is easier to get things right at this stage than to try and fit it in later. But they also said that of all the solutions they looked at (and they looked at all of them), OpenNMS was the most “open”. It wasn’t in the sense that OpenNMS is truly free and open source, but that OpenNMS doesn’t force you into a particular way of management. It enabled them to configure the system to manage their network the way they wanted to, and not the way some vendor thought they should.

It warmed my heart.

I travel too much. I missed the snow back home, I miss my wife, and I’m now in yet another hotel room. But it’s meetings like today that make it all worthwhile.

barcampESM

A couple of years ago I went to my first barcamp, barcampRDU. A barcamp is advertised as an “un-conference”. Instead of setting up a schedule of speakers, a bunch of people get together and sort of build the speaker list from the attendees. The part I like most about barcamp is hangin’ with folks between presentations. It’s something that is a little harder to do at a conference you pay to attend since no one wants to miss out on the presentations (or at least they pretend not to want to miss them).

I can’t remember if it was whurley or John Willis who first brought barcampESM to my attention. I want to think it was both: whurley brought it up at LinuxWorld but John Willis helped make it happen with his enthusiasm for the project. The idea was to get a bunch of end users together with a bunch of vendors in a barcamp-style setting, but the reality turned more into just a bunch of vendors. Despite that I had a lot of fun, although I felt very much like an outsider most of the time. The reason follows.

We had two of the “big four” in the room, BMC and Tivoli (IBM), and one of the “little four”, Zenoss. All of them sell software. The big guys are trying to figure out how to get their lower end stuff maintained for less, and the little guys are just trying to sell software into the space being vacated by the big guys. Both see “open source” as the solution to their problems. If the big guys could build a community around the low end stuff they could focus their resources on expensive high-end solutions and software, and the little guys can’t build a market for their “enterprise” software unless they get some mindshare by releasing a small subset of their code as “open”.

You might think I’m being a little harsh here, but let’s be real. The big guys sell to the C-level, and old-school management isn’t sexy enough to catch their attention any more. They need new stuff like “Business Services Management” etc.

The little guys are all financed by VCs, and in order to get that money they had to produce a business plan with revenue projections – projections based on selling software licenses. So how will such a project respond to a user who implements the “enterprise” functionality as open source? I don’t think they could accept the code and commit it while at the same time satisfy shareholders who expect software revenue. They’d have to make up some excuse about why they couldn’t commit it, and unless the independent community was strong enough to fork the code, nothing would change.

So I was a bit of a duck out of water. I don’t know the first thing about selling software, or what needs to be done to make it easier to sell. But I do know network management, and so did most of the attendees, so I felt comfortable talking to them.

John Willis is a big Tivoli user, and he brought in two other heavy hitters, Doug McClure and Heath Newburn. Both guys were really cool and really knowledgeable. There is no denying the appeal of an end-to-end framework, but in practice it has been extremely hard to realize. I take a bit of pride in the fact that OpenNMS is being used on some huge networks, but these guys are managing some ginormous networks, since only the huge companies can afford ‘em (grin). I’m certain I can learn a lot from these guys.

Speaking of heavy hitters, I got to see Doug Stevenson (Dougie Fresh) for the first time in many years. Doug has forgotten more about network management than I will ever know, and he’s had experience with pretty much everything out there. We used to call him the Elvis of network management.


Dougie and Me

He’s doing well, currently working on Tivoli, and after a heart attack in 2004 he’s in the best health I’ve seen.

Oh, nice lead in for a little story. When Doug was at AOL, he worked for a guy named Scott, who used to work for me. I was on the phone with Scott back in 2001 when he had a heart valve tear. He didn’t know he had Marfan syndrome and it had caused his valve to weaken, and he had to be airlifted to the nearest large hospital for emergency surgery and a valve replacement. When I talked to him a few days later, a had to say, “Scott, I know I like to run my mouth, but dude, next time just say you have to go.”

Of course Doug had to call Scott after his heart attack to let him know he wouldn’t be in for awhile. For once I know there was a manager who understood completely what one of his guys was going through.

Anyway, back to barcampESM.

Coté showed up and introduced me to Michael Nels and Kartick Suriamoorthy from Alterpoint/Ziptie. I really want to check out Ziptie now. It’s a configuration management tool written in Java, and there might be some integration points that we could leverage.

All I need is time. Ah, what I’d give for a 36 hour day (as long as everyone else stayed on 24 hour days).


Me, Frank and Chris

I got to meet Charles Crouch from Red Hat (JBoss). He is working on the Red Hat fork of the Hyperic code base that JBoss licensed a few years ago. And I also got to meet Frank Sheiness (archon~ from the #opennms IRC channel) and Chris Bowman from Korcett, but more on them later.

I’m sure I’m missing other people. Bill Karpovich and Erik Dahl from Zenoss were there, and Erik has promised to wear an OpenNMS polo if I send him one. I got to hear how they got started with the “Zen of Open Source Software”. Chip Holden from BMC was on a panel with Erik, Heath and me moderated by John.

Man, I really didn’t get to spend as much time with everyone as I wanted.

There was a little bad news. As whurley was riding his skateboard toward the venue with a bunch of gear on his back, a woman walked out right in front of him. When he planted his foot to stop, the weight on his back shifted and caused his ankle to twist. After hobbling around for most of the day he went to the doctor and came back on crutches. They are not sure if it is broken or not.

Overall I was glad I came. The crowd was cool and I really like Austin. Many thanks to BMC and Zenoss for footing the bill for lunch and dinner. Also, check out the new Open Management Consortium site. It looks like they are breathing new life into the project (but don’t participate in the stupid “What’s your favorite open source monitoring tool?” poll – perhaps it will get less like Tiger Beat as the site gets going).

Austin City Limits

I’ve been to Texas at least a hundred times. Now, if you subtract out those trips that just involved the DFW airport, it’s still over twenty, but not once have I been to Austin.

Growing up in North Carolina, I grew up watching Austin City Limits on PBS and many of the people I know in Texas speak of Austin in glowing terms, so I’ve always wanted to visit.

The opportunity came when I was invited by whurley to visit his barcampESM event. We don’t have much of a budget for events like this, but as luck would have it I have to be in San Antonio next week, so I figured I could fly to Austin, visit barcamp, and then drive down.

I was a little confused as to when the event would occur, since there wasn’t much information on the web site, and all it said for the longest time was “18-20 January in Austin”. I didn’t want to make my travel plans at the last minute due to higher costs, so seeing that there was a direct flight from RDU to AUS the evening of the 17th, I figured I’d just flight out then and if nothing was going on during Friday I’d just visit some companies in the Austin area.

Austin is a hotbed of technology companies, with some large established companies and lots of startups. I was able to visit three of them and they were all within a couple of miles of each other in an area just north of the city.

RenewData was my first stop. They have a really interesting business. They escrow electronic documents such as e-mail for clients, and should those clients get subpoenaed in a court case, they are able to retrieve the relevant information to comply with the court order.

Their data center is more secure than many I have been in, since the information contained within could be very sensitive. I can’t say much more than that since every visitor is required to sign an NDA.

The next stop was the Austin studio of Bioware. I don’t play many computer games, but one of the last ones I bought was Neverwinter Nights, so I was jazzed about getting to spend some time there.

A successful video game can gross more than a blockbuster hollywood movie (e.g. Halo 3) and with the advent of broadband to the home the business model has expanded to include subscription-based online play. In order to support these massively multiplayer online role-playing game (MMORPG) environments, a lot of server and network resources are required. If they are slow to respond or unavailable, it directly impacts the user experience and can result in players leaving the game and a loss of revenue.

Bioware is working with LucasArts to create what will be one of the largest MMORPGs ever made, and thus their management solution must be able to scale. I can’t tell you much more beyond that. In this case it is not due to an NDA but due to the fact that the guys wouldn’t give me any details (and, yes, I did plead). I assume it will be Star Wars related. They did take me to lunch though at Trudy’s. Since I live in an area of North Carolina with a large Latino population, I take my Mexican food seriously, and although Mexican food is not Tex-Mex I had a wonderful lunch. I love being in a state where avocados are plentiful.

My last visit was to NCSoft. We’ve been working with these guys for a number of years, and it was a nice continuation of the day’s “gaming” theme. If I can only get to Blizzard I should have the industry covered. (grin)

NCSoft produces games such as City of Heroes/City of Villains, the ever popular Guild Wars series and Tabula Rasa. I believe they are the largest independent online gaming company in the world. The lobby was pretty cool:

I travel to a lot of places, and the vibe at NCSoft was one of the best I’ve experienced lately. Everyone seemed excited to be there. Of course, as with all the best companies they use OpenNMS to monitor their network (grin).

I got to meet a bunch of people (and for you Tabula Rasa fans I walked by Richard Garriott’s office) and then we went out to Waterloo for drinks. Ah, Shiner on tap.

The one theme I noticed throughout the day was the presence of a services model. All three of these companies based their main revenue on providing services. As a pure services company, OpenNMS is often criticized, at least within the investment community, but here are examples of services companies that are thriving. Of course the issue of scalability is always at the top of the list, which is why OpenNMS can be a good fit. OpenNMS was designed to monitor tens of thousands of services from a single instance. Other solutions scale by adding servers at a much lower number, and while you can run OpenNMS in a distributed fashion, that need is usually only required in the largest of networks. You don’t really want to have so many servers that you need a management solution for your management solution.

My last meeting of the day was one I’ve looked forward to for some time. I came across Michael Coté’s blog last year, and we’ve been keeping up an e-mail correspondence since then. As avid travelers and members of the American Airlines AAdvantage program who work to maximize our miles, we had some things in common, and since Coté is a man who likes to listen and I am a man who likes to talk, it was a good fit.

He took me to Artz Rib House, which reminded me of some of my favorite BBQ places back home. With live music in the background, good food and Shiner on the table, it was a nice ending to a fun day. We talked pretty bluntly about the industry and it was nice to spend time with someone who wasn’t focused just on management. He promised to introduce me to some people such as the folks at Alterpoint, and I got to talk about my determination to put “free” back into “open source software”.

When he dropped me back at the hotel we ran into whurley in his Incredible Hulk-green Scion B, where he was dropping off Doug McClure and John Willis (one of the barcampESM organizers). The latter two disappeared into the elevator before I could catch ‘em, but I would see them again the next day at barcampESM.