Prodigal Customers

Growing up in the southern United States meant Sunday mornings were spent at Sunday School. One of the stories we would study was the Parable of the Prodigal Son. A man has two sons. The younger son asks for his inheritance in advance and he goes off and squanders it. When he returns, his father throws a big celebration to welcome him back.

I never really got the point of that story, as I always identified with the older, dutiful son, so it is surprising that it took working with OpenNMS for me to understand it.

We have great customers. Since we do little marketing, before we get a customer they have to first discover OpenNMS, then investigate it to see if it meets their needs, and only then do they contact us. It means that they are self-selecting, and without exception they are incredibly smart, physically beautiful and possessing of a wit so sharp they make Ginsu knives look dull. (grin)

The first company to ever buy an OpenNMS support subscription did so in December of 2001, and this year they renewed for the 17th time. It is a wonderful testament to the work of the team that they created something to inspire such a long commitment.

That said, we do lose a few customers each year. The first one I lost was a little heartbreaking. It was a hospital in Virginia, and when I called them to see if they would renew their support subscription they told me “no”. I was a little shocked, as I was unaware of any problems and they hadn’t opened tickets in awhile, and they told me that was the point. They loved OpenNMS but it “just worked” so they saw no value in getting support, they were still using it.

A more common case for us losing a customer is that our “internal champion” leaves. OpenNMS is a complex and powerful tool, and it does take awhile to climb the learning curve to see its full potential. If all of that knowledge is focused on one person, and that person leaves, their replacement can be overwhelmed and seek out something simpler, even if it is more expensive and less powerful.

I am alway saddened when this happens, but lately we’ve been experiencing what I’m calling “Prodigal Customers”. These are customers who leave and come back.

Cartoon by Chad Essley http://www.cartoonmonkey.com

I love them, and always want to slaughter (figuratively) the fattened calf to welcome them back.

It’s hard to explain, but while it is wonderful to have someone use something you’ve created for almost two decades straight, it is almost more rewarding to have someone go and try something else and discover it doesn’t stack up. Heck, I’d love it if all our customers could try out every possible option, because those that then chose OpenNMS for their solution would truly recognize what an awesome platform it can be.

Being 100% open source, OpenNMS does not have any way to “lock in” a particular customer. You can use it with our services or without, but you always have access to the latest code. Thus choosing to use OpenNMS is a validation of the work we’ve put into it, and whether you are a long time customer, a new customer, or a “prodigal” customer, your preference to use OpenNMS makes all the work to create it worthwhile.

2017 Australian Network Operators Group (AusNOG) Conference

I am excited to be returning to Australia for the third time next week. This trip is to speak at the Australian Network Operators Group (AusNOG) annual conference in Melbourne.

AusNOG Promo Graphic

I can’t believe I’ve gone for so long and not known about Network Operator Groups (NOGs). There are quite a few of them and I think they would be a perfect audience to introduce to the OpenNMS Project. One of our users on the OpenNMS chat server is from Perth and he made me aware of the conference, and I was humbled and delighted to have my presentation accepted.

At OpenNMS we strive very hard to separate the project (.org) from the commercial entity that supports the project (.com) and this presentation will be strictly focused on the project. It’s a wonderful thing about OpenNMS: if it meets your needs, cool. If not, also cool. I just want more people to be aware of open source options, especially in the carrier and enterprise space.

And it looks like open source is definitely making inroads at AusNOG. The talk before mine is about Ansible and Salt. There is another talk on using open source to build a version of NetNorad, and another one on open source for big data analytics. The one after mine is about modern network monitoring, so I hope I tick at least a few boxes on his list.

I hope to see you there (although it looks like it is sold out) but let me know if you are in the area and perhaps I can at least say “hi”.

Freedom Feud

My official title at OpenNMS is CEO, but I’ve worn several hats in the last 12+ years, including accountant, receptionist, HR manager and janitor. Now I get to add record producer to the list.

I guess it all started back in 2012. OpenNMS was doing pretty well and I wanted us to give a little something back to the community. As a fan of MC Frontalot I came across his FAQ and found out that you could actually book him for things like conferences, kids parties, bar mitzvahs and the like. We were sponsoring the Ohio Linuxfest (by the way, I’m a keynote speaker there this year along with the ever amazing Karen Sandler) and I decided to see if I could book him to play a show. Turns out he is pretty affordable (for contrast, Henry Rollins starts out at $10,000 per event, which isn’t unreasonable but doesn’t count as affordable for us just yet). I booked him to play a solo gig and finally got to meet the man. He did a great show, everyone seemed to enjoy it, and we became friends of a sort.

While Front is very much a nerd, he had not had much exposure to free software. A lot of musicians rely on Windows-based software to create their music (Front mainly uses Reaper and Professor Kliq is such an Ableton fan he has their logo tattooed on his wrists) and thus they aren’t used to using open source. The OLF event went so well I hired him a few more times, and I think it was at SCaLE when I suggested he write a free software song. His reply was, well, why don’t you commission one.

Front is talent for hire. He did a couple of tracks for New Relic, “Nerd Life” and “Small Data“, and while we don’t have anywhere near the budget of that company we felt that free software deserved to be examined under his lyrical microscope, so we started the process.

Note that this was a couple of years ago, back in 2015, so it took awhile. Front and I had a number of conversations about free software and I started him on his education. I pointed him to the works of Richard Stallman and Eric S. Raymond (notably The Cathedral and the Bazaar), as well as Lawrence Lessig and organizations such as the Free Software Foundation (FSF) and the Electronic Frontier Foundation (EFF). We also talked about the classic “free as in beer” vs. “free as in freedom” confusion that arises out of the term “free software”, which ended up forming the basis of the song.

Later in 2015 I wanted to do something special for the OSCON show in Portland. This time I decided to hire Front along with his band. In the previous shows he’d done for us it was him and “DJ CPU” providing the music, and while those were great shows I was unprepared for the “live band” experience. It took it to another level. During that show Front performed a bit of the song, but it wasn’t until last year’s All Things Open that the whole song was played for the first time (again with the band, since, awesome).

With the song almost complete we now how to figure out how to present it to the world. I wanted a video, so I decided to turn to animator Chad Essley. Chad had done the video for “Shudders” off of Question Bedtime and we had gotten to know each other through a fund raising promotion he did for the EFF where we sponsored adding OpenNMS references to that video. I felt he could do a good job with it, since he is both talented and he gets the subject matter.

Now when I said “record producer” above I basically meant I signed the checks, but it was cool watching artists such as Front and Chad work out even small details when it came to the video. Animation can take a really long time, so we debated on combining it with some live action, etc., to both speed up the process and reduce the cost. It was at this point that I was introduced to the concept of a “lyric video”.

Ed Sheeran had just come out with a new song, and in order to promote it as quickly as possible he released a video that pretty much consisted of just the song’s lyrics. While at this point in our process we had a portion of the animation completed, I thought that adding the lyrics to it would both speed things up as well as improve the experience, as Front’s rhymes on this track are some of the tightest he’s ever written. The end result is both a video that is fun to watch as well as one that gets the message across in an entertaining manner.

It seems to have been well received, and as I write this it has had over 10,000 views and mostly positive comments, and we got a shout out on Boing Boing.

Speaking of lyrics, the phrase that has gotten the most comments is the line “Pull down capitalism till it’s rubble and chain”. When asked about it, Front commented:

I thought it was funny to equate Free Software with that dirty commie yearning for collectivized well-being and shared responsibility! Those two realms of thought are not directly in line with each other as far as I can tell. But I imagined the kind of business dude who is reflexively distrustful of free software, and I pictured him saying, “you mean… LIKE COMMUNISM??” Thought I’d give him a little dig at the end there.

I can’t remember if I shared my story with Front when we were first talking about creating this song, but when I got started with OpenNMS I was called a communist a number of times on various message boards. It confused me, since it came mainly from people who made their living as network management consultants. I was thinking, hey, here is a tool that lets you provide better solutions for your customers while showcasing your unique talents via your ability to deliver them, and that’s communism? Sounds like good business to me. But there is still that element of “anti-capitalism” associated in free software (I blame the phrase “so you can help your neighbor” in the Free Software Definition, but that’s just me).

The whole process was a lot more work than I thought it would be, but I’m very pleased with the result. Check out the video as there are a lot of in-jokes and Easter eggs, and I’ve been told that “floppy head Lawrence Lessig” was met with approval by the man himself.

Oh, in the spirit of free software, Front has published the song under the Creative Commons (CC BY-SA) license, and I am hoping to see a few cool remixes come out of this. I’ve reached out to both Professor Kliq and Raccoon Fink and if they find time to play with it, I’ll be sure to let you know (and let me know if you find some out there). Front is working on a new album tentatively entitled The Internet Sucks so maybe this track will make the cut.

Rhythmbox: Repeat One Song

I use Linux Mint as my default desktop environment. One of the reasons I started using it was that the default applications for many functions were the default applications I would choose if I were making a distro.

Their choice for music player used to be Banshee. I really liked Banshee – it reminded me of the early versions of iTunes before that application became too complex. Unfortunately, Banshee is no longer under active development, and the last release was back in 2014.

As the underlying libraries have changed and matured, Banshee has not kept up. For example, if I plugged in my handy Banshee would hang if the MTP mount was being accessed elsewhere. Mint recently decided to switch to Rhythmbox, and I’ve finally made the decision to start using it.

One of the things I’ve learned about open source is to be patient learning a new app. The reason there are often numerous open source solutions for various tasks is that people do things differently, and it can take awhile to understand how a particular application is designed to work. I’ve found that many features I thought were lacking in Rhythmbox were there, just implemented differently than I expected. If the feature is, indeed, missing, you can often add it with a plugin.

I’ve recently been exposed to the music of Imogene Heap, starting with her album Sparks. I really like the sixth track “Lifeline” and I wanted to listen to it a couple of times on repeat. There is a repeat button on the menu, so I clicked it, but that just repeats the playlist. In other apps you can click that icon multiple times and it will rotate through various options: i.e. repeat playlist, repeat song, etc. Not so with Rhythmbox.

A quick search and I found a plugin hosted on Github to add this feature. I downloaded the repository, unzipped the file, and then copied it to ~/.local/share/rhythmbox/plugins/. I then went to Tools -> Plugins and enabled “Repeat One Song” (no restart of the app needed). Now, under the Edit menu, I have the option to repeat the current song.

Repeat One Song Screenshot

Not quite as nice or intuitive as clicking on a button, but it works.

While I see this as a great example of the awesomeness of open source, it also brought out the downside of free software. There was this comment:

This should not be a plugin.. It should be there by default if rhythmbox wants to call itself a music player.

Seriously? A bunch of people write a complex piece of software, give it away for free, build in a way to extend it, but no, that’s not enough. This guy isn’t satisfied that these folks didn’t cater to his every need, even though edumucelli has gone to the trouble to add it.

Free software isn’t a free solution, and I just wanted to post this to remind people, including myself, that often it takes an investment of time to really get to understand how an application works.

In open source, often our first goal is to make something that works before we make something that is easy to use. I’m not proud of this, but quite frequently the motivation behind the developers of free software is to solve a problem important to them and it just happens to be useful to others. And even companies that focus heavily on UI and try to build intuitive interfaces can get it wrong. I’ve had to work with recent versions of iTunes and find it rather difficult to do simple things, although I’m certain that if I used it more I would learn what I needed to do, just like I have with Rhythmbox.

Which I’ve grown to like. It works well with my mobile device and I’m eager to watch it improve even more in the future.

When Not To Start an Open Source Company

Over the weekend, Chris Aniszczyk posted a link on Twitter to a very interesting article by Matt Klein about his decision not to start an open source company around his project, Envoy. I thought it raised a number of interesting points worth a few comments.

First off, Matt works for Lyft, which, in case you haven’t heard of it, is Uber without the moral decay. I abandoned Uber some time ago, despite being an early adopter, and I’ve been very happy with Lyft. One of the main differences is that Lyft allows you to tip your driver, which I almost always do with few exceptions. The fact that Lyft is able to keep and motivate people like Matt speaks volumes for their corporate culture.

It also demonstrates a wonderful trend of commercial companies starting and maintaining open source projects. I’ve been working with open source for almost two decades and I can remember when any software developed at a company was considered confidential. To this day there are a number of vendors who consider their SNMP MIB files (which, I should point out, are really only useful to people who have purchased their products) proprietary information. Companies like Lyft, Paypal and Facebook, none of which would self-identify as open source companies, have gained a lot of value for little cost by making the tools they use open source.

When talking about open source for the enterprise, I often talk about the fact that it is the processes that a company uses to serve its customers that make it unique and define its value, not the tools used by the company. So often with commercial software you have to change those process to fit how the application thinks you should work, and in the process you lose some part of what makes you special to your customers. With open source you can fit the application to those processes. It is how you use the tools and not the tools themselves that is important, and so there is a lot to gain and little to lose by making them open source.

Getting back to Matt’s article, he is a project maintainer for Envoy, which is a “high performance C++ distributed proxy and communication bus designed for large service oriented architectures.” While I don’t consider myself a coder so I don’t claim to fully understand the its advantages, I do recognize enough buzzwords in that sentence to know that it would attract some attention from investors, and Matt was approached about leaving Lyft to start a commercial business around Envoy. He decided not to, and as I read his article about his decision I realized I’d found a kindred soul, someone who was more interested in creating something of value that would last versus making a quick buck.

He had me with this paragraph:

In my opinion, the best opportunity to commercialize OSS lies with projects that can be easily turned into SaaS products. Ultimately, even if software is completely open, many customers are happy to pay for a turnkey solution that “just works” and has a defined SLA with 24/7 monitoring and support. In some sense, customers pay for the operational expertise that comes from deeply understanding and running the software, versus the software itself.

Amen.

I’ve been making a living on open source for 15 years now working with OpenNMS, and I’ve spent a lot of time thinking about business models. We started out with the “service and support” model, which kept the doors open but limited growth. Then our clients started asking us for features, so we added custom development, which was time intensive but allowed us to finance OpenNMS features which attracted even more customers as the platform became more powerful. When we hit the problem of trying to balance the “release early, release often” philosophy of open source with the need for stability, we adopted the Red Hat model of splitting our application into a feature-rich, rapidly developed release (which we call Horizon™, similar to Fedora) and a more stable, subscription-based release that may lag in features but is better suited production environments (which we call Meridian®, similar to RHEL). But ultimately we came to the decision that what we really wanted to do was to offer OpenNMS as a service.

One company that inspired that decision was Automattic, maintainers of WordPress. I don’t think I know of a more powerful piece of software that is easier to install. They have a famous “5 Minute Install” that is quite simple. First, you drop the software into the webroot of your web server of choice. Next, you create a database account on your database of choice with certain permissions. Then you navigate to a web page and follow the prompts.

However, for a lot of people, terms like “webroot” are gibberish, and even with WordPress you still need some minimal database skills to maintain it. So Automattic offers up WordPress as a service. For a small monthly fee they’ll do everything for you, and this has generated revenues on the order of tens of millions of dollars per year.

OpenNMS is way more complicated, thus the value of a hosted version should be greater. In order to do so we needed some way to access the client’s network in a secure fashion, so with Horizon 20 we introduced the Minion. The Minion software allows for OpenNMS functionality to be distributed. It is built on the Karaf container, so once installed all of its features can be remotely managed. For smaller networks, the Minion can be sold as an appliance and talk to a hosted version of OpenNMS. It can bring a complex and powerful tool like OpenNMS into the hands of the masses.

For larger companies it solves issues of scale as Minions can be deployed to cover even the largest networks (our goal is IoT scale). We’ve had them in production at one client for months now handling over 2 million events an hour. That translates to around 555 events per second, although the system itself can handle over 10,000 events per second so they have room to grow. If they ever hit that limit, we can simply add more Minions. They have the option of hosting all of OpenNMS in their own data center, or they could choose a hybrid model where some of the functionality is outsourced.

For pretty much the first time in the history of OpenNMS, we are seriously and actively seeking investment. There are a number of companies entering this space who have raised enormous amounts of money, and we think we can be competitive for far less money and provide a better solution. Plus, also for the first time in the history of OpenNMS, we have a reason to make it easier to use versus spending all of our resources making it more powerful.

Matt talks about investment in his post (remember Matt? As usual, I’ve made this all about me. Meeee!) It was actually his stories about dealing with investors that prompted me to write this. As Envoy started to get some traction, investors wanted him to leave and start a company. He writes:

Over the last few months I’ve been told by several investors that no OSS has become ubiquitous without having explicit commercial backing. I think this is false and is situation dependent. If anything, I would argue that if I were to leave Lyft now and start a platform company around Envoy, it will decrease the chance of Envoy becoming ubiquitous, primarily because it would negate all of the reasons laid out above.

That first sentence is interesting, since “ubiquitous” and “commercial” are a little vague. I would make the claim that the Apache web server was ubiquitous until its success spawned NGINX, and it was backed by the Apache Software Foundation which is a non-profit. Is a foundation “commercial”? The idea that for a project to become successful it needs a number of people to spend a lot of time working on it seems obvious, and the best way to achieve that is to pay those people to work on it.

He goes on to write:

It took me a lot of time to ultimately understand the previous simple point. Investors are extremely persuasive. They capitalize on “fear of missing out.” However, it’s important to realize that the opportunity cost is hugely mismatched between investor and company.

When he writes “investors” above I believe he means specifically venture capitalists. We’ve talked with a few VCs in the past and I can remember the almost “strong arm” tactics they used. If I hear “a rising tide lifts all boats” one more time, I might have to hit somebody. I’m not saying that all VCs are the same, but many of them come across as gamblers and not investors. I’m risk friendly but I don’t gamble. I’m heavily invested in wanting to build something with OpenNMS that outlasts me (it is already much bigger than me as the team I work with has way more to do with its success than I do) and I don’t want to gamble with it.

I do hope that there are some investors out there that can appreciate that aspect of our company as well as the fact that we’re profitable, have mature products and wonderful customers. Perhaps private equity or perhaps another company that shares our vision and wants to advance the project through acquisition. In any case we’re looking for them.

When I was a young man, old guys like I am now would tell me “work on something you love, not just for the money”. I always dismissed it with the thought that with enough money I can buy love. When you immerse yourself in something as personal as an open source project for ten to twelve hours a day, year after year, you really do have to love it and the satisfaction you get just can’t be bought. Matt’s thoughts are similar:

Ultimately, on a personal level I’m just having too much fun solving tough computer science problems at large scale at Lyft and building a community around Envoy. The bar to do something different is therefore extremely high, and it took a long time to realize that it’s perfectly OK to accept that and keep going down the existing path that I’m on. On another level, leaving now to start a company would feel very much like not following through on my original goal of open sourcing Envoy; the industry desperately needs a high quality and community-driven solution to microservice networking. Follow-through is something I take very seriously.

With that attitude the success of Envoy is almost assured.

How Version 2.0 Killed Android Wear

I am the happy owner of an LG Urbane smartwatch. Unfortunately, I just upgraded to Android Wear 2.0 and now I can’t use it.

Andrea Wear 2.0 Upgrade

Luckily for me, my smartwatch is not “mission critical”. If I leave it at home by mistake, I don’t turn around to go back to get it. The main thing I use it for is notifications. I like the fact that if it is with me, it will automatically mute my phone and then vibrate when I have a notice. A quick glance at my wrist will tell me if I need to deal with it right this moment, or if it can wait.

The second thing I use it for is to do simple voice searches or to set reminders and timers. Outside of that there are a few apps I use and I like the fact that it tracks my steps, but overall I don’t use a ton of features.

When the notice popped up that I could upgrade, I blindly went ahead and did it. In retrospect, that was stupid, but I often get in trouble rushing out to install the “new shiny”. The upgrade seemed to go fine, and I didn’t think that much about it until lunch.

One of the things I do before heading out to lunch is check the temperature to see if I need a jacket. So I did the usual wrist flick to “wake” the watch and said “Ok Google” to get to the voice prompt.

Nothing happened.

Hrm, I did some research and apparently with 2.0 you have to press the button on the side of the watch to get to the Google prompt. I think this is a huge step backward, because now I have to involve both hands, and I find it ironic that with Android Wear 1.5 I I had to sit through a demo of one-handed gestures over and over again (I often have to re-pair my watch due to reloading software on my phone) and now they’ve thrown “do everything with one hand” out the window.

Anyway, I pressed the button which then brought up the Google Assistant setup screen on my phone. With 2.0 if you want to use voice searches, etc., you must use Google Assistant and you have to give Google access to all of your contacts, calendars etc.

(sigh)

I work hard to “sandbox” my Google activity from the rest of my digital life. It’s not that I think they are evil, it’s just that I don’t want anyone to have that much information on me, well, other than me. I kind of despair for free and open source software solutions in the consumer space. Everyone seems to be rushing to adopt these “always on” digital assistants with absolutely no regard to privacy, and this is causing vendors to lock down their ecosystems more and more. While open source is definitely winning on the server side, I don’t think the outlook has ever been so grim on the consumer side.

There were some upsides with 2.0, such as improvements to the look and feel, but I also found that I didn’t care for the new notification system (I seemed to miss a lot of them – perhaps I needed to change a configuration). But the requirement for Google Assistant was a deal breaker.

I thought about going back to 1.5, which I liked, but I can’t seem to find a factory image. In trying to locate one, I discovered that TWRP does have a version for bass (the codename for the LG Urbane) and I should have installed that and made a backup before upgrading. I contacted LG and they told me it was impossible to downgrade. That’s a load of crap because I could easily sideload the old version if they made it available, but then I’d have to deal with constant upgrade reminders and the few apps I do use would probably stop support for 1.5 to focus on 2.0.

It just isn’t worth it.

I know at least one of my three readers is thinking I should just cave and learn to embrace the Google, but I can’t bring myself to do it. I am eagerly awaiting open source alternatives like Asteriod OS (which just isn’t ready for daily use) and Mycroft (which is supposed to be shipping units this month) but I really don’t think I’ll miss my Urbane enough to spend the time on it.

I plan to sell my Urbane on eBay and I’ve gone back to my previous “dumb” watch (a nice little Frederique Constant I bought on a flight from Dubai to London). It’s kind of a shame since I enjoyed using it, but to be honest I’m not going to miss it all that much.

The Importance of Contributor Agreements

One thing that puzzles me is the resistance within the open source community to contributor agreements. This was brought into focus today when I read that the OpenSSL Project wants to migrate to the Apache 2.0 license from the current project specific OpenSSL license.

In order to do that they need permission from all of the nearly 400 contributors of the project over the last 20+ years, and contacting them will be a huge undertaking. If one person refuses to agree, then they will either have to abandon the effort, or locate that person’s contribution and either remove or replace it.

Many years ago we found out that a company was using OpenNMS in violation of our license. When our lawyer approached them about it, they claimed that they were only using those parts of the code for which we didn’t hold copyright. At that time, early versions of OpenNMS were still copyright Oculan, the company that started the project, and not OpenNMS. Since Oculan wasn’t around anymore it took us awhile to track down the intellectual property, but in the end David and I were able to mortgage our houses to purchase that copyright so that now the project can control all of the code and defend it from license abuse in the future.

But the question arose about what to do moving forward, specifically how should we deal with community contributions? In the past companies like MySQL required all contributors to sign a document with phrases like “You hereby irrevocably assign, transfer, and convey to MySQL all right, title and interest in and to the Contribution” which seemed a little harsh.

I posed this question to the Order of the Green Polo, the de facto project administrators, and DJ Gregor suggested we adopt the Sun Contributor Agreement that we now call the OpenNMS Contributor Agreement, or OCA. This was a straightforward document that asked two things.

First, you attest that you have the right to contribute the code. This is more important than you know, because it helps remove liability from the project should the contribution turn out to be encumbered in some way, such at the author writing it as part of their job and thus it is actually the property of the employer. We allow both individuals and companies to sign the OCA.

Second, you assign copyright to OpenNMS while retaining copyright yourself. This introduces the concept of “dual copyright”. Now some critics will say that this concept hasn’t been tested in court, but there is a long history of authors sharing copyright. Considering that Oracle maintained the agreement in the form of the Oracle Contributor Agreement, it appears that their lawyers were satisfied.

I claim responsibility for the license under which these Contributor Agreements are published: the Creative Commons Attribution-Share Alike License. When DJ suggested the Sun Contributor Agreement I noticed that there wasn’t any license on the agreement itself. I didn’t want to just copy it and change “Sun” to “OpenNMS”, so I contacted Brian Aker who had just moved to Sun with the MySQL acquisition and asked him about it. Soon thereafter the Agreement was updated with the license and we adopted our version of it.

Once we adopted the OCA, I was tasked with tracking down anyone who had ever contributed to OpenNMS outside of the company or Oculan and asking them to sign it. They all did, but I can tell you that I had a hard time tracking down a number of them (people move, e-mails change). I don’t envy OpenSSL at all.

I hope this story illustrates the importance of some sort of Contributor Agreement for open source projects. They don’t have to be evil, and in the end getting your copyright and licensing issues completely sorted out will make managing them in the future so much easier.

2017 Europe: Riga

Latvia is the 39th country I’ve been able to visit, and based on Riga it is easily in my top ten. I really enjoyed my short time here.

Getting off the bus from Tallinn, the first thing I noticed was that it was a little colder here. Both Helsinki and Tallinn are right on the water, but Riga is slightly inland. Still, it wasn’t a hard walk from the bus station to the hotel, and I got to see some of the Old Town.

Frozen Stream in Riga

I had the rest of the day to myself, so I decided to explore the City. One thing I noticed about Riga is that it is very clean. Granted, when you have piles of snow that don’t melt this doesn’t mean everything looks brand new, but I didn’t see the usual trash and paper on the ground like I might find in London or Paris. While the buildings may be old, they are well maintained, and some are quite beautiful, which is not how I imagined a former Soviet bloc country to look.

Riflemen Monument

Granted, there were a few reminders, such as the impressive “Riflemen Monument“. This was originally meant to honor those in the Latvian military who supported the Bolsheviks (the “red” riflemen) but I was told that now it also honors the opposition “white” riflemen.

The reason I came to Riga was to participate in a conference held by LATA (Latvijas atvērto tehnoloģiju asociācija or the Lativian Open Technology Association). LATA is a volunteer organization with only one employee, Ieva Vitolina, who was kind enough to invite me to speak.

Not only were the people in general in Riga very kind to me, the LATA people treated me like a diplomat.

Main Entry Hall for the LATA Conference

Before the conference I was introduced to Jānis Treijs, of the LATA Board. A very nice man, Jānis is very tall, and I had to joke that when I studied physics we used to say all people were two meters tall to make the math easier, but it is rare I actually get to meet someone that tall.

LATA conference room

The conference was held at the Latvijas Universitātes Dabaszinātņu akadēmiskais centrs (Latvian University of Natural Sciences Academic Center) which was a very modern facility, much nicer than many of the schools I attended in my youth. The morning program was held in this main room, and after lunch we would break out into another room as well (which was where my talk was to be made). About half of the program was in Latvian, with the other half in English.

IBM was a sponsor, and Andrzej Osmak from Poland gave a talk on IBM’s approach to open.

Andrzej Osmak

To be quite frank, OpenNMS would not exist without IBM. They are a main supporter of the Apache Foundation and most of the developers use Eclipse as their IDE. The only small criticism I would have about that talk was an emphasis on permissive licensing. I think permissive licenses are great in the proper context, but they aren’t the best choice for everyone.

This was followed by another talk in English by Dr. John O’Flaherty from Ireland.

John O’Flaherty

His focus was on “open data” and the different levels with which data can be made available. I am always amazed at what wonderful things people can create when companies and governments make data available in a usable fashion, and John gave several examples of those.

The remaining morning talks were in Latvian, so I just tried to understand them through the slides. The Clusterpoint presentation was interesting in that the slides were in English but the presentation itself was given in Latvian.

The morning ended with an awards presentation which had three categories: the most open institution, the most substantial contribution to technology promotion, and the best start-up.

Then it was the lunch break, which I spent talking about business and free software with Valdis, Ieva’s husband. It was then time to get ready for my own presentation.

There were two presentations in English about open source business. Including mine, Aleksejs Vladiševs the founder of Zabbix shared his experiences. It was kind of ironic that both of us work at pure open source companies and both of us work in the network monitoring space. Despite that, we tend not to compete, and it was interesting to see how similar our paths were.

My talk seemed well received, although I had a little less than 30 minutes so I didn’t have any time for questions. I was humbled that the winner of the LATA start-up award, Mihails Scepanskis, wanted to ask me some questions about open source business afterward, and along with his wife Anna and Vladis, we spent pretty much the rest of the conference talking. As usual, my favorite conference track turned into the “Hallway Track” once again.

National Library

That evening, the organizers of the conference took a group of us on a tour of the National Library of Latvia. This is a major landmark in Riga and it is easy to spot from many places in the city. It was planned for many years, but finally opened in 2014.

National Library Sign

The interior hosts a 400+ seat state of the art theatre, but the first thing I noticed was the central atrium.

National Library Atrium

Inside it there is a wall of books. These were books donated by the Latvian people to the library, and it stretches for several stories. We were also told an interesting story, when the library opened several thousand books were moved from the old location to the new building via a “human chain“. People formed a line over a mile long and passed the books hand to hand.

National Library Book Wall

The tour took us up through the building, and we got to see a number of the large (and not so large) reading rooms. One that caught my eye was dedicated to American culture.

National Library American Culture Room

I found it interesting that the books on display included ones by Noam Chomsky, James Carville and articles from the New Yorker.

Each floor was color-coded, and we were told that the colors corresponded to the “pre-Euro” Latvian currency, the Lat. The higher floors had colors that corresponded to higher denominations.

National Library

At the top was an interesting display. It was a Cabinet of Folksongs. This wooden cabinet holds over a quarter of a million Latvian folksongs written on small slips of paper.

Cabinet of Folksongs

The tour was followed by a wonderful meal in a restaurant in the Library itself. I got to spend more time talking with Aleksejs, Jānis, his wife and John, as well as drinking some nice beer over wonderful food.

The next morning Jānis’s wife had arranged for me to meet with the ITC department of the City Council of Riga. Riga firmly believes in Internet access for its population. The City has more free WiFi coverage than any other European City, and the Council is responsible for providing as many services as possible to its citizens to make sure the government is responsive to their needs. It was a refreshing conversation. They use a number of tools, including Zabbix, so I wasn’t expecting them to switch to OpenNMS, but I had a nice meeting learning about their environment and sharing a little bit about OpenNMS.

Corner House

We had a little time before lunch, so we made a quick visit to the “Corner House“. This was a beautiful apartment building that was taken over by the Cheka, a division of the KGB, and was the source of terror for many citizens of Latvia as late as 1991. It reminded me of the House of Terror in Budapest. Jānis’s wife told a story of her mother having to go to this building for an interview as the Cheka was interested in one of her relatives.

Corner House

It is a shame that a thing of such beauty could be used for such evil.

After that we met up with Jānis for a wonderful meal, and then I made my way to the airport for my trip to Brussels for FOSDEM.

As the airBaltic Q400 took off and got above the clouds, the cabin was suddenly filled with light. I realized that I had not seen the sun properly in a week. If Riga and its people can be this beautiful in the dark of winter, it must be a truly magical place in the summer. I hope one day soon to return.

2017 Europe: Helsinki

I am spending a week touring the eastern side of Europe, with the first stop being two nights in Helsinki. I should end up in Brussels next weekend for FOSDEM, and I am looking forward to my first time at that conference.

I’m here because I was invited to speak at an open tech conference in Riga, Latvia, and I couldn’t resist the invitation. Riga is home to Zabbix, a company very much like OpenNMS in that we both do network monitoring and we are both 100% open source. One might think this would make us enemies – quite the contrary. For some reason we really get along and also, for some reason, we rarely compete.

In trying to find a route from North Carolina to Latvia, I noticed a number of choices went through Helsinki. I had been to Helsinki once and really enjoyed it (despite it being winter). I also remembered from that trip that Finland is very close to both Russia and Estonia. You can be in St. Petersburg in three hours by train or Tallinn in two hours by ferry.

It was my goal to visit 50 countries by the time I turned 50 years old. I didn’t make that goal (I got to 37), but I figured I could use this trip to both visit Estonia and Latvia, adding two to the list.

My first flight out of RDU was canceled, so they routed me through JFK. I arrived in Helsinki three hours later than planned, but my bag made it with me so it worked out. It was dark and sleeting, but it wasn’t too difficult to take the new train into the city center and find my hotel.

HSL Train Helsinki Airport

I always like coming to Finland because it was the home of Linus Torvalds. Now I know he has lived in the US for many years and I also know he didn’t invent the idea of free software, but I still feel some sort of homecoming when I arrive since I doubt OpenNMS would be here if it weren’t for Linus.

There is an awesome company in Helsinki that is also an OpenNMS customer, so I was able to spend Monday visiting with them. Due to an NDA I can’t name them, but they are doing some amazing work in this part of the world. I got to learn more about their business as well as to share where we are going with OpenNMS.

Like many of our larger clients, they have an inventory system that they have integrated with OpenNMS in order to manage their monitoring needs. Since that system also contains customer relationships (which equipment is used to provide network services for particular clients of theirs) we played around with the Business Service Monitor (BSM). They should be able to export their network information into OpenNMS to create a customer impact topology, so that when there is an issue they can quickly determine the root cause. It is exactly why we created the feature and I’m eager to see how they use it.

They are also interested in using the Minion feature due in Horizon 19. This should allow them to easily deal with overlapping address space and any scalability concerns, plus they should be able to get rid of their current “manager of managers” solution. Exciting times.

They are looking to hire, so if you are in the area and have OpenNMS experience, send me your CV and I’ll be happy to forward it on to them.

Ulf and Hacienda Napoles at Liberty or Death

That evening, Ulf and I managed to indulge our taste for vintage and craft cocktails with a visit to Liberty or Death. This is a bar near my hotel that serves amazing cocktails in a very relaxed atmosphere. It was a nice ending to a very good day.

Ferry Terminal Statue

The next day will find me on a ferry boat to Tallinn. I don’t know of any OpenNMS users in Estonia, but I am still eager to see the city.

OpenNMS 101

One of my favorite things to do is to teach people about OpenNMS. I am one of the main trainers, and I usually run the courses we hold here at OpenNMS HQ. I often teach these classes on-site as well (if you have three or more people who want to attend, it can be cheaper to bring someone like me in for a week than to send them here), and the feedback I got from a recent course at a defense contractor was “that was the best class I’ve ever attended, except for the ones I got to blow stuff up.”.

Unfortunately, a lot of people can’t spare a week away from the office nor do they have the training or travel budget to come to our classes. And teaching them can be draining. While I can easily talk about OpenNMS for hours on end, it is much harder to do for days on end.

To help with that I’ve decided to record the lessons in a series of videos. I am not a video editing wizard, but I’ve found a setup using OBS that works well for me and I do post production with OpenShot.

The first class is called “OpenNMS 101” and we set it up as a video playlist on Youtube. The lessons are built on one another so beginners will want to start with Module 0, the Introduction, although you can choose a particular single episode if you need a refresher on that part of OpenNMS.

My goal is to put up two or three videos a week until the course material is exhausted. That will not begin to cover all aspects of OpenNMS, so the roadmap includes a follow up course called “OpenNMS 102” which will consist of standalone episodes focused on a particular aspect of the platform. Finally, I have an idea for an “OpenNMS 201” to cover advanced features, such as the Drools integration.

I’ve kept the videos as informal as the training – when I make a mistake I tend to own it and explain how to fix it. It also appears that I use “ummmmmmm” a lot as a place holder, although I’m working to overcome that. I just posted the first part of “Module 4: Notifications” and I apologize for the long running time and the next lessons will be shorter. I had to redo this one (the longest, of course) as during the first take I forgot to turn on the microphone (sigh).

We have also posted the slides, videos and supporting configuration files on the OpenNMS project website.

I’d appreciate any feedback since the goal is to improve the adoption of OpenNMS by making it easier to learn. Any typos in the slides will be fixed on the website but I am not sure I’ll be able to redo any of the videos any time soon. I think it is more important to get these out than to get them perfect.

Perfection is the enemy of done.