2017 Dev-Jam – Day 3

By Day 3 we’ve settled into a rhythm, so I don’t have much to report from the actual OpenNMS side of things. Personally, I spent way too much time trying to figure out why Twitter is blocking links to this blog. It’s been ad-free for over a decade, yet Twitter thinks it is spam.

We believe it was because we were using dlvr.it to post things I write here to the OpenNMS Twitter feed and thus it got flagged as automation (which is, apparently, bad). I can understand it, but my complaint is that there is no clear process for getting it resolved. I think I’ve submitted the proper request and I’ve even tried to back-channel some help through friends of friends, but I think I’m just going to focus on posting on Google Plus from here on out, unless I need to complain. (grin)

So the gang worked while I bitched at Twitter. Oh, I do have a picture of a Canadian food product: ketchup flavored potato chips, modeled by Jeff.

Dev-Jam: Ketchup Potato Chips - photo credit Jessica

I can report on what we did in the evening. Usually Dev-Jam involves seeing a Major League Baseball game, specifically the Minnesota Twins. However, the only MLB team ever in Montréal, the Expos, played their last season here a year before Dev-Jam started (2004).

However, Montréal does host an international fireworks competition, so we got tickets. It’s held at La Ronde, which is an amusement park in the Six Flags chain. La Ronde is located on the northern end of St. Helen’s Island (Île Sainte-Hélène). While the park itself wasn’t very crowded, it turns out that Metallica fans, including our own Alejandro, were descending on the southern part of the island for a concert.

To get there from Grey Nuns, we took the Metro. The Metro station on St. Helen’s Island is near a museum called The Biosphere, complete with a geodesic dome created by Buckminster Fuller. Might be worth a return visit.

Dev-Jam: Montréal Biosphere - photo credit Mike

We took a free shuttle to get to the park, and our tickets also granted us admission. Out of habit I had taken my grandfather’s pocket knife, not realizing they would have metal detectors, but they had a cool system where I could drop it off and pick it up later (so I didn’t have to hide it in the bushes).

As soon as we got into the park, the sky opened up and poured on us. Some of us found shelter under building overhangs, some on rides and some just got wet. One of the rides was the carousel, where Ben, Seth and Cynthia seemed to have fun.

Dev-Jam: La Ronde Carousel - photo credit Mike

Luckily, the rain didn’t last too long. Since it was National Hot Dog Day, we decided to find some. As a fan of the show Silicon Valley, I suggested we try out the “Not Hotdog” app. Seems to work.

Dev-Jam: Not Hotdog - screenshots Mike

Afterward there was just enough time for a ride on Le Vampire.

Dev-Jam: Le Vampire Rollercoaster - photo credit Chris

Jessica is in the back of this shot, with Ronny and Jesse in the front.

Then it was time to see the show. There are three seating sections: Bronze, Silver and Gold. Gold had free beer but we had been told that the Silver section (Argent) had the best views. I wasn’t disappointed. And this being Quebec, there had to be clowns.

Dev-Jam: Fireworks Clowns - photo credit Ben

In section 307 we were on an elevated platform looking out over a lake. The fireworks were launched from the other side of the lake and synchronized with music. As this is an international competition, the presenter for our show was Germany. They played the German national anthem and introduced the performers.

Dev-Jam: Fireworks Introductions - photo credit Ben

The show … it was amazing.

And when I say it was amazing, I mean it was like I had never seen fireworks before, and I’ve seen them at places like EPCOT at Disney World. Since we were pretty close to the launch site, it was like they took up your whole field of vision. Plus, I never knew there were so many different types. They used the whole area, from high overhead down to the lake itself (at one point in time it was if the water was ablaze in green fire). Shells would explode overhead in a burst of color, followed by a second color and when those faded there would be a ring left that slowly faded away.

Dev-Jam: Fireworks Above - photo credit Ben

Sometimes the sky would just be a fountain of gold, and at other times the action would be at ground level. At times I felt they were telling a story. Not exactly one I understood, such as the red fountain thingies seem to be fighting the shooty yellow thingies, but it still evoked an emotional response.

Dev-Jam: Fireworks Down Low - photo credit Ben

The emotional high point for me was when they played Pachelbel’s Canon. I know it is cliché but the “Canon in D” is one of the most beautiful things ever made, and to see it illustrated in fire was simply breathtaking.

Speaking of things to take your breath away, soon after the show started the air was filled with smoke and ash from the fireworks. While it really didn’t detract from the show, people with health issues related to breathing should think carefully about attending.

Dev-Jam: The Bridge at Night - photo credit Ben

The show was 30 minutes long, and by the time we headed out to leave so did 40,000 Metallica fans. This was further complicated by the bridge to the island being closed so people could watch the fireworks. Rather than waiting for buses to arrive once the bridge was reopened, we decided to walk. There was a great view from the top, and the bridge itself was lit in green.

Dev-Jam: View from Bridge - photo credit Ben

We got back to the dorm around midnight, having had a great time. I hope I get a chance to see the fireworks show again, perhaps at next year’s Dev-Jam.

2017 Dev-Jam – Day 2

Dev-Jam was fully underway by Tuesday morning, starting with another Canadian tradition, Tim Hortons.

Dev-Jam: Tim Hortons Box

Lots of great discussions were going on. Ronny demonstrated Project Atlas, more formally known as the GraphML Topology Provider. This allows you to use GraphML to create topologies within the OpenNMS user interface.

Dev-Jam: Ronny Doing a Demo of Atlas

There is also a topology.xml file on the OpenNMS Forge github repository that can be used as an example.

Jesse gave us a demonstration of Project Helm. This is a Grafana plugin that let’s you combine fault and performance data from multiple instances of OpenNMS on one dashboard.

Dev-Jam: Jesse Doing a Demo of Helm

While it is just in alpha, the goal is to let users manage alarms directly from the dashboard, including acknowledging them, adding “sticky” and journal notes, etc. We have been working for years now on making a robust ReST interface for OpenNMS and it is really paying off by allowing us to create features like this. Since all of the communication between Grafana and the OpenNMS system (or systems) is via ReST, there is no need to store and manage data locally.

Dev-Jam: Helm Screenshot

If you want to play with Helm, you should be running the latest Horizon 21 snapshot.

We have a person named Roberto attending Dev-Jam for the first time and I was eager to find out why he was interested in OpenNMS, so I spent some time talking with him. His company deploys underwater fiber-optic cable. Their customers used to be large telecommunications providers, but now they deal mainly with very large Internet companies, and those companies are requesting a higher level of monitoring information. It was one of those “very large Internet companies” that suggested they use OpenNMS, and it was interesting to learn about the challenges of running and managing undersea fiber.

Only a small part of the cable contains the fiber as most of it consists of a thick protective sleeve. The sleeve has to be thicker near shore since there is a greater chance of damage from things like ships’ anchors. Also, electrical current flows through the sleeve which attracts sharks, who then proceed to bite the cable. Here’s a video:

I’m eager to see how they end up using OpenNMS.

As I mentioned before, we are staying in the Grey Nuns Residence at Concordia University. According to Wikipedia there is a crypt in the basement where nearly 300 bodies are buried, most of them nuns who had lived at the Grey Nuns Motherhouse. It is off limits to visit, but I wanted to see if I could at least find the entrance.

Our conference is being held in a large room called E104, and most of the rooms of the people attending are also on the east side of the residence. My room, however, is on the west side and to get there I have to walk about 200 meters (it is a big place). You go out of E104, down to the basement and along a very long corridor before heading up several floors.

Along this corridor you will see a nondescript door,

Dev-Jam: Door to Grey Nuns Crypt

and if you peek through the little round window you can see into the crypt.

Dev-Jam: Image of Grey Nuns Crypt

The graves are marked with plain wooden crosses, and the one nearest the door died in 1885, although there are certain to be much older graves in the crypt. Apparently there was a project to move the bodies out of Grey Nuns but the government balked due to the fact that some of the people buried there died of infectious diseases (the history of the Grey Nuns [pdf] confirms that several nuns died of the Spanish Flu of 1918).

It is a pretty solemn place and in stark contrast to the rest of the dormitory.

Speaking of things definitely not solemn, for dinner we all headed to a Japanese restaurant nearby called Kinka Izakaya. Izakaya means a pub, and the menu consists of lots of small plates, kind of like Japanese tapas.

Dev-Jam: Dinner at Kinka Izakaya

The place met a number of criteria: good food, can seat 24 people and close to the dorm. We also had to try a “Sake Bomb” in which a small amount of sake is suspended over a glass of beer. You then drop the sake into the beer and drink. Yes, there is video:

Good times.

2017 Dev-Jam – Day 1

Dev-Jam is an unstructured conference. Our goal is to simply put a bunch of incredibly smart people in a room and see what happens. That said, we do officially start and end the conference. On Monday morning we get together to make introductions and to talk about projects that we want to pursue during the week. This allows people with similar interests to work together if they want. On Friday we have presentations on what got accomplished.

Dev-Jam: People Around a Table

I usually start off the week, and then turn it over to Jesse White (our CTO and GM of The OpenNMS Group Canada).

Dev-Jam: Jesse White in an MC Frontalot shirt

I thought it was cool that he was wearing an MC Frontalot T-shirt as we commissioned him to produce a free software song that we released on Independence Day (July 4th).

Another cool thing about OpenNMS is that we try to work as transparently as possible. While a lot of projects allow public access to their git repository, I believe OpenNMS is the only one that has a repository for every branch that automatically builds packages as commits are made (the list can be found at http://yum.opennms.org/repofiles/ but be patient as there are so many it can take a minute or so to load). We also publish a weekly newsletter called “This Week in OpenNMS” (or TWiO). This week Ben posted some ideas bouncing around this year’s Dev-Jam which include:

  • updating packaging (yum/Debian) infrastructure including better support of upstream PostgreSQL packages
  • improved wifi link support in Enlinkd and topology
  • improving the opennms.com and opennms.org web sites
  • simplifying collection of OCAs
  • integrating DigitalOcean and Xen requisition tools
  • Spark chat integration
  • ReST infrastructure improvements
  • discussing how to improve Docker image generation
  • grafana dashboard for UPS data
  • northbound interfaces for Drools (scriptd-like interface for alarms)
  • structured data monitor (using the XML collector infrastructure)
  • Cisco ACI integration
  • OSGi deployment of ReST services
  • donut charts on the front page
  • rewrite the node list page
  • migrate documentation to AsciiBinder
  • trigger IFTTT events when alarms change
  • porting the Go version of the minion to a new platform

Dev-Jam: Laptop and Can of Cheerwine

I should note that Ben is also a fan of that North Carolina export, Cheerwine.

Dev-Jam: People Meeting to Discuss the Website

Jessica, our graphic designer, pulled together a meeting to discuss our web presence. We recently revamped the opennms.com website and we are looking to determine improvements needed for the opennms.org website.

Speaking of Jessica, she also designed our Dev-Jam shirts.

Dev-Jam: Front of Dev-Jam Shirt

The front is meant to represent summer camp.

Dev-Jam: Back of Dev-Jam Shirt

The top symbol is for Montréal, the bottom is Ulf the OpenNMS mascot, the tag to the left represents coding and the right image is for fireworks (we are attending the competition on Wednesday).

It’s not all work. Recently I read about a restaurant very close to Concordia (where we are holding the conference) that offered free meals to people who need them. I wanted to support that, so for lunch a group of us went to Marché Ferdous, which had been written up on sites such as CNN, the BBC and Huffington Post.

Dev-Jam: Marché Ferdous Entrance

It’s a small shop a couple of blocks away from our dorm, and I got the falafel platter.

Dev-Jam: Marché Ferdous Falafel Platter

My meal was about CAD$10 so I paid with a $20 bill and told them to keep the change. Always gotta pay it forward, yo. (grin)

The falafel was just okay (I’m spoiled as I get to eat Angie’s falafel on a regular basis – some of the best falafel on the planet and I should know as I’ve had it pretty much all over the planet) but the sides were excellent. Everyone else got meat and really enjoyed it.

After lunch we took a side trip to a SAQ store to check out the spirit selection. Later that evening there was a tasting …

Dev-Jam: Monday Whiskey Selection

… which probably had something to do with the decision to screen Strange Brew, eh?

Dev-Jam: Strange Brew on Screen

2017 Dev-Jam – Day 0

♬ It’s the most wonderful time of the year ♬

It’s hard for me to believe that we are getting ready for our twelfth annual OpenNMS developers conference, Dev-Jam.

Dev-Jam: Welcome to Montreal Sign

This year we changed venues from our normal spot at Yudof Hall at the University of Minnesota to Concordia University in Montréal, Quebec, Canada. We have to plan these things out months in advance, and back in January there was talk of greatly increasing the effort required to enter the United States, especially for visitors from other countries, requiring them to, among other things, reveal social media passwords. Since a large portion of people attending Dev-Jam come from outside the US, we thought it prudent to move the conference. Plus, The OpenNMS Group now has a corporation in Canada, so it also seemed to be a nice way to mark that development.

After searching around for a place to hold the event, we settled on the Grey Nuns Residence, a large dormitory. While the individual rooms are not as nice as Yudof, the conference space is really large and should work out well. Plus, Grey Nuns is considered one of the most haunted places in Montréal, although so far no one has reported anything unusual.

We have over 20 people attending this year, which is down a little bit from normal. We have several people from Minneapolis who attend, and by moving it to Canada it became difficult for them to make it. In an ironic twist our friend Muthu from India was unable to get his visa to Canada approved in time to make the conference. But outside of some weather delays everyone else made it here safely.

So did the Cheerwine. It has become a Dev-Jam tradition for me to bring the North Carolina made cherry-flavored soda to share with everyone, and sometimes it gets a little, soggy, in transit. All 48 cans made it to Montréal, although it won’t last all that long.

There is a bit of adjustment to being in Quebec. I get to practice my (poor) French, and I love the fact that it is like everything has subtitles (legally, English should appear under the French in no more than half the size). Plus we’re having to get used to things distinctly Canadian such as Thrills chewing gum. Flavored with rosewater, the taste has been compared to soap, a fact that is proudly displayed on the box.

Dev-Jam: Thrills Gum Box

Another Montréal tradition is poutine. This is a dish of thin french fries topped with cheese curds and gravy.

Dev-Jam: Poutine

The one I tried was a variation that included chicken and a whiskey BBQ sauce. This was from a restaurant chain called St-Hubert which specializes in chicken (seriously, the set menu offered chicken, chicken, chicken, chicken, ribs, chicken, chicken, and, you guessed it, chicken). It was good, and it was nice to find a place to serve 20+ people comfortably.

Dev-Jam: Dinner at St. Hubert

While change can be challenging, I think this is going to be a great week. Outside of David and myself, both Mike and Craig have made all twelve Dev-Jams, and DJ has made all but one. One of my goals with OpenNMS is to built something that lasts, and it is nice to have traditions that have continued for this long.

Freedom Feud

My official title at OpenNMS is CEO, but I’ve worn several hats in the last 12+ years, including accountant, receptionist, HR manager and janitor. Now I get to add record producer to the list.

I guess it all started back in 2012. OpenNMS was doing pretty well and I wanted us to give a little something back to the community. As a fan of MC Frontalot I came across his FAQ and found out that you could actually book him for things like conferences, kids parties, bar mitzvahs and the like. We were sponsoring the Ohio Linuxfest (by the way, I’m a keynote speaker there this year along with the ever amazing Karen Sandler) and I decided to see if I could book him to play a show. Turns out he is pretty affordable (for contrast, Henry Rollins starts out at $10,000 per event, which isn’t unreasonable but doesn’t count as affordable for us just yet). I booked him to play a solo gig and finally got to meet the man. He did a great show, everyone seemed to enjoy it, and we became friends of a sort.

While Front is very much a nerd, he had not had much exposure to free software. A lot of musicians rely on Windows-based software to create their music (Front mainly uses Reaper and Professor Kliq is such an Ableton fan he has their logo tattooed on his wrists) and thus they aren’t used to using open source. The OLF event went so well I hired him a few more times, and I think it was at SCaLE when I suggested he write a free software song. His reply was, well, why don’t you commission one.

Front is talent for hire. He did a couple of tracks for New Relic, “Nerd Life” and “Small Data“, and while we don’t have anywhere near the budget of that company we felt that free software deserved to be examined under his lyrical microscope, so we started the process.

Note that this was a couple of years ago, back in 2015, so it took awhile. Front and I had a number of conversations about free software and I started him on his education. I pointed him to the works of Richard Stallman and Eric S. Raymond (notably The Cathedral and the Bazaar), as well as Lawrence Lessig and organizations such as the Free Software Foundation (FSF) and the Electronic Frontier Foundation (EFF). We also talked about the classic “free as in beer” vs. “free as in freedom” confusion that arises out of the term “free software”, which ended up forming the basis of the song.

Later in 2015 I wanted to do something special for the OSCON show in Portland. This time I decided to hire Front along with his band. In the previous shows he’d done for us it was him and “DJ CPU” providing the music, and while those were great shows I was unprepared for the “live band” experience. It took it to another level. During that show Front performed a bit of the song, but it wasn’t until last year’s All Things Open that the whole song was played for the first time (again with the band, since, awesome).

With the song almost complete we now how to figure out how to present it to the world. I wanted a video, so I decided to turn to animator Chad Essley. Chad had done the video for “Shudders” off of Question Bedtime and we had gotten to know each other through a fund raising promotion he did for the EFF where we sponsored adding OpenNMS references to that video. I felt he could do a good job with it, since he is both talented and he gets the subject matter.

Now when I said “record producer” above I basically meant I signed the checks, but it was cool watching artists such as Front and Chad work out even small details when it came to the video. Animation can take a really long time, so we debated on combining it with some live action, etc., to both speed up the process and reduce the cost. It was at this point that I was introduced to the concept of a “lyric video”.

Ed Sheeran had just come out with a new song, and in order to promote it as quickly as possible he released a video that pretty much consisted of just the song’s lyrics. While at this point in our process we had a portion of the animation completed, I thought that adding the lyrics to it would both speed things up as well as improve the experience, as Front’s rhymes on this track are some of the tightest he’s ever written. The end result is both a video that is fun to watch as well as one that gets the message across in an entertaining manner.

It seems to have been well received, and as I write this it has had over 10,000 views and mostly positive comments, and we got a shout out on Boing Boing.

Speaking of lyrics, the phrase that has gotten the most comments is the line “Pull down capitalism till it’s rubble and chain”. When asked about it, Front commented:

I thought it was funny to equate Free Software with that dirty commie yearning for collectivized well-being and shared responsibility! Those two realms of thought are not directly in line with each other as far as I can tell. But I imagined the kind of business dude who is reflexively distrustful of free software, and I pictured him saying, “you mean… LIKE COMMUNISM??” Thought I’d give him a little dig at the end there.

I can’t remember if I shared my story with Front when we were first talking about creating this song, but when I got started with OpenNMS I was called a communist a number of times on various message boards. It confused me, since it came mainly from people who made their living as network management consultants. I was thinking, hey, here is a tool that lets you provide better solutions for your customers while showcasing your unique talents via your ability to deliver them, and that’s communism? Sounds like good business to me. But there is still that element of “anti-capitalism” associated in free software (I blame the phrase “so you can help your neighbor” in the Free Software Definition, but that’s just me).

The whole process was a lot more work than I thought it would be, but I’m very pleased with the result. Check out the video as there are a lot of in-jokes and Easter eggs, and I’ve been told that “floppy head Lawrence Lessig” was met with approval by the man himself.

Oh, in the spirit of free software, Front has published the song under the Creative Commons (CC BY-SA) license, and I am hoping to see a few cool remixes come out of this. I’ve reached out to both Professor Kliq and Raccoon Fink and if they find time to play with it, I’ll be sure to let you know (and let me know if you find some out there). Front is working on a new album tentatively entitled The Internet Sucks so maybe this track will make the cut.

Rhythmbox: Repeat One Song

I use Linux Mint as my default desktop environment. One of the reasons I started using it was that the default applications for many functions were the default applications I would choose if I were making a distro.

Their choice for music player used to be Banshee. I really liked Banshee – it reminded me of the early versions of iTunes before that application became too complex. Unfortunately, Banshee is no longer under active development, and the last release was back in 2014.

As the underlying libraries have changed and matured, Banshee has not kept up. For example, if I plugged in my handy Banshee would hang if the MTP mount was being accessed elsewhere. Mint recently decided to switch to Rhythmbox, and I’ve finally made the decision to start using it.

One of the things I’ve learned about open source is to be patient learning a new app. The reason there are often numerous open source solutions for various tasks is that people do things differently, and it can take awhile to understand how a particular application is designed to work. I’ve found that many features I thought were lacking in Rhythmbox were there, just implemented differently than I expected. If the feature is, indeed, missing, you can often add it with a plugin.

I’ve recently been exposed to the music of Imogene Heap, starting with her album Sparks. I really like the sixth track “Lifeline” and I wanted to listen to it a couple of times on repeat. There is a repeat button on the menu, so I clicked it, but that just repeats the playlist. In other apps you can click that icon multiple times and it will rotate through various options: i.e. repeat playlist, repeat song, etc. Not so with Rhythmbox.

A quick search and I found a plugin hosted on Github to add this feature. I downloaded the repository, unzipped the file, and then copied it to ~/.local/share/rhythmbox/plugins/. I then went to Tools -> Plugins and enabled “Repeat One Song” (no restart of the app needed). Now, under the Edit menu, I have the option to repeat the current song.

Repeat One Song Screenshot

Not quite as nice or intuitive as clicking on a button, but it works.

While I see this as a great example of the awesomeness of open source, it also brought out the downside of free software. There was this comment:

This should not be a plugin.. It should be there by default if rhythmbox wants to call itself a music player.

Seriously? A bunch of people write a complex piece of software, give it away for free, build in a way to extend it, but no, that’s not enough. This guy isn’t satisfied that these folks didn’t cater to his every need, even though edumucelli has gone to the trouble to add it.

Free software isn’t a free solution, and I just wanted to post this to remind people, including myself, that often it takes an investment of time to really get to understand how an application works.

In open source, often our first goal is to make something that works before we make something that is easy to use. I’m not proud of this, but quite frequently the motivation behind the developers of free software is to solve a problem important to them and it just happens to be useful to others. And even companies that focus heavily on UI and try to build intuitive interfaces can get it wrong. I’ve had to work with recent versions of iTunes and find it rather difficult to do simple things, although I’m certain that if I used it more I would learn what I needed to do, just like I have with Rhythmbox.

Which I’ve grown to like. It works well with my mobile device and I’m eager to watch it improve even more in the future.

A Brief History of an Open Source Company

I’ve been invited to give a keynote at this year’s Ohio Linuxfest being held in Columbus, Ohio, on 29-30 September. I am both excited and humbled as this is one of my favorite conferences of the year and I know a lot of amazing people will be there to share their knowledge of free and open source software.

Ohio Linuxfest Logo

I take my presentations pretty seriously, especially keynotes, so I wanted to come up with something that was both funny and interesting. They asked me to speak on running a business around open source software, and I immediately thought I should come up with some click-bait title like “Ten Things About Open Source Business, Four of Them Will Shock You!” but it just didn’t feel right. Then I thought about Hawking’s A Brief History of Time and that seemed more fitting.

My most popular talk so far has been on starting an open source business, but that focuses mainly on the mechanics of the process. For this talk I want to trace my history with OpenNMS starting with my first day on the job and then describing how it grew to become what it is today. In those 15+ years I’ve had a lot of adventures, some good and some bad, and I’ve met a some wonderful people. It is the work of many of those people that actually make OpenNMS what it is – I act more like a “crap umbrella” with my one job being to block all of the things that might keep the team from being productive – and I want to talk how that came about. This presentation will consist almost entirely of real world examples of the problems we encountered and our decision process for solving them.

I hope it will be entertaining and useful, and look forward to seeing you there.

When Not To Start an Open Source Company

Over the weekend, Chris Aniszczyk posted a link on Twitter to a very interesting article by Matt Klein about his decision not to start an open source company around his project, Envoy. I thought it raised a number of interesting points worth a few comments.

First off, Matt works for Lyft, which, in case you haven’t heard of it, is Uber without the moral decay. I abandoned Uber some time ago, despite being an early adopter, and I’ve been very happy with Lyft. One of the main differences is that Lyft allows you to tip your driver, which I almost always do with few exceptions. The fact that Lyft is able to keep and motivate people like Matt speaks volumes for their corporate culture.

It also demonstrates a wonderful trend of commercial companies starting and maintaining open source projects. I’ve been working with open source for almost two decades and I can remember when any software developed at a company was considered confidential. To this day there are a number of vendors who consider their SNMP MIB files (which, I should point out, are really only useful to people who have purchased their products) proprietary information. Companies like Lyft, Paypal and Facebook, none of which would self-identify as open source companies, have gained a lot of value for little cost by making the tools they use open source.

When talking about open source for the enterprise, I often talk about the fact that it is the processes that a company uses to serve its customers that make it unique and define its value, not the tools used by the company. So often with commercial software you have to change those process to fit how the application thinks you should work, and in the process you lose some part of what makes you special to your customers. With open source you can fit the application to those processes. It is how you use the tools and not the tools themselves that is important, and so there is a lot to gain and little to lose by making them open source.

Getting back to Matt’s article, he is a project maintainer for Envoy, which is a “high performance C++ distributed proxy and communication bus designed for large service oriented architectures.” While I don’t consider myself a coder so I don’t claim to fully understand the its advantages, I do recognize enough buzzwords in that sentence to know that it would attract some attention from investors, and Matt was approached about leaving Lyft to start a commercial business around Envoy. He decided not to, and as I read his article about his decision I realized I’d found a kindred soul, someone who was more interested in creating something of value that would last versus making a quick buck.

He had me with this paragraph:

In my opinion, the best opportunity to commercialize OSS lies with projects that can be easily turned into SaaS products. Ultimately, even if software is completely open, many customers are happy to pay for a turnkey solution that “just works” and has a defined SLA with 24/7 monitoring and support. In some sense, customers pay for the operational expertise that comes from deeply understanding and running the software, versus the software itself.

Amen.

I’ve been making a living on open source for 15 years now working with OpenNMS, and I’ve spent a lot of time thinking about business models. We started out with the “service and support” model, which kept the doors open but limited growth. Then our clients started asking us for features, so we added custom development, which was time intensive but allowed us to finance OpenNMS features which attracted even more customers as the platform became more powerful. When we hit the problem of trying to balance the “release early, release often” philosophy of open source with the need for stability, we adopted the Red Hat model of splitting our application into a feature-rich, rapidly developed release (which we call Horizon™, similar to Fedora) and a more stable, subscription-based release that may lag in features but is better suited production environments (which we call Meridian®, similar to RHEL). But ultimately we came to the decision that what we really wanted to do was to offer OpenNMS as a service.

One company that inspired that decision was Automattic, maintainers of WordPress. I don’t think I know of a more powerful piece of software that is easier to install. They have a famous “5 Minute Install” that is quite simple. First, you drop the software into the webroot of your web server of choice. Next, you create a database account on your database of choice with certain permissions. Then you navigate to a web page and follow the prompts.

However, for a lot of people, terms like “webroot” are gibberish, and even with WordPress you still need some minimal database skills to maintain it. So Automattic offers up WordPress as a service. For a small monthly fee they’ll do everything for you, and this has generated revenues on the order of tens of millions of dollars per year.

OpenNMS is way more complicated, thus the value of a hosted version should be greater. In order to do so we needed some way to access the client’s network in a secure fashion, so with Horizon 20 we introduced the Minion. The Minion software allows for OpenNMS functionality to be distributed. It is built on the Karaf container, so once installed all of its features can be remotely managed. For smaller networks, the Minion can be sold as an appliance and talk to a hosted version of OpenNMS. It can bring a complex and powerful tool like OpenNMS into the hands of the masses.

For larger companies it solves issues of scale as Minions can be deployed to cover even the largest networks (our goal is IoT scale). We’ve had them in production at one client for months now handling over 2 million events an hour. That translates to around 555 events per second, although the system itself can handle over 10,000 events per second so they have room to grow. If they ever hit that limit, we can simply add more Minions. They have the option of hosting all of OpenNMS in their own data center, or they could choose a hybrid model where some of the functionality is outsourced.

For pretty much the first time in the history of OpenNMS, we are seriously and actively seeking investment. There are a number of companies entering this space who have raised enormous amounts of money, and we think we can be competitive for far less money and provide a better solution. Plus, also for the first time in the history of OpenNMS, we have a reason to make it easier to use versus spending all of our resources making it more powerful.

Matt talks about investment in his post (remember Matt? As usual, I’ve made this all about me. Meeee!) It was actually his stories about dealing with investors that prompted me to write this. As Envoy started to get some traction, investors wanted him to leave and start a company. He writes:

Over the last few months I’ve been told by several investors that no OSS has become ubiquitous without having explicit commercial backing. I think this is false and is situation dependent. If anything, I would argue that if I were to leave Lyft now and start a platform company around Envoy, it will decrease the chance of Envoy becoming ubiquitous, primarily because it would negate all of the reasons laid out above.

That first sentence is interesting, since “ubiquitous” and “commercial” are a little vague. I would make the claim that the Apache web server was ubiquitous until its success spawned NGINX, and it was backed by the Apache Software Foundation which is a non-profit. Is a foundation “commercial”? The idea that for a project to become successful it needs a number of people to spend a lot of time working on it seems obvious, and the best way to achieve that is to pay those people to work on it.

He goes on to write:

It took me a lot of time to ultimately understand the previous simple point. Investors are extremely persuasive. They capitalize on “fear of missing out.” However, it’s important to realize that the opportunity cost is hugely mismatched between investor and company.

When he writes “investors” above I believe he means specifically venture capitalists. We’ve talked with a few VCs in the past and I can remember the almost “strong arm” tactics they used. If I hear “a rising tide lifts all boats” one more time, I might have to hit somebody. I’m not saying that all VCs are the same, but many of them come across as gamblers and not investors. I’m risk friendly but I don’t gamble. I’m heavily invested in wanting to build something with OpenNMS that outlasts me (it is already much bigger than me as the team I work with has way more to do with its success than I do) and I don’t want to gamble with it.

I do hope that there are some investors out there that can appreciate that aspect of our company as well as the fact that we’re profitable, have mature products and wonderful customers. Perhaps private equity or perhaps another company that shares our vision and wants to advance the project through acquisition. In any case we’re looking for them.

When I was a young man, old guys like I am now would tell me “work on something you love, not just for the money”. I always dismissed it with the thought that with enough money I can buy love. When you immerse yourself in something as personal as an open source project for ten to twelve hours a day, year after year, you really do have to love it and the satisfaction you get just can’t be bought. Matt’s thoughts are similar:

Ultimately, on a personal level I’m just having too much fun solving tough computer science problems at large scale at Lyft and building a community around Envoy. The bar to do something different is therefore extremely high, and it took a long time to realize that it’s perfectly OK to accept that and keep going down the existing path that I’m on. On another level, leaving now to start a company would feel very much like not following through on my original goal of open sourcing Envoy; the industry desperately needs a high quality and community-driven solution to microservice networking. Follow-through is something I take very seriously.

With that attitude the success of Envoy is almost assured.

CubaConf 2017

UPDATE: Today the United States administration announced tougher restrictions on travel to Cuba. While nothing has changed at the moment, there will be some changes in the next 30 days. This should not impact people attempting to go to Cuba for this conference as it should fall under the “professional” or “educational” travel categories. This may change again before November and I’ll be sure to post updates.

While tourist travel remained officially banned, Obama also allowed a broad category of “people to people” visits to Cuba. Trump’s new directive still allows individual travel in all but that category, and reverts to an earlier policy of requiring “people to people” visits only in a Treasury-licensed group.

Free and open source software is as close to a true meritocracy than anything else I’ve found. It doesn’t matter what is the color of your skin, your gender or where you live, your value is judged simply by your contributions to the project. I wrote up my favorite instance of that for opensource.com concerning my friend Alejandro who got involved with OpenNMS when he lived in Venezuela. He and his wife are now permanent residents in the US due to his work on our project.

I actually forget how I came across CubaConf, but I was immediately interested in attending. This is an annual free software conference held in Havana, Cuba.

CubaConf

It has been illegal for US Citizens to travel to Cuba since before I was born. Last year the Obama administration eased some of those restrictions, so it is now possible, under certain conditions, to travel to Cuba as well as to use US Dollars while there.

Cuba has been pretty isolated since the 1960s, and as it races to catch up with the rest of the world it will need access to modern technology, especially software. I see an opportunity for free software to play a huge role in the future of that country, and I am eager to meet the people who will help make that happen.

I want to use this post to encourage all of my free and open source software friends to come to CubaConf. This is a three-day event that follows a format similar to one we used for our OpenNMS user conferences. The first day is a normal conference, with various tracks and presentations set to a schedule. The second day is a “barcamp” style conference where the attendees will set the agenda, and the third day is a hackathon.

Presentations are welcome in both Spanish and English, so I’ve submitted two talks (both in English). One is on starting an open source business. This will be different from my usual talk as I want to focus on how someone in Cuba could both spread the use of free software while getting paid to do it, without as much focus on setting up a corporation or other formal business entity. The second talk is on OpenNMS. While business transactions are still difficult between the US and Cuba, I really want to bring the magic that is OpenNMS to their attention so that when things ease between our countries people will be familiar with it.

I plan to attend all three days, and Alejandro is coming with me to help with any language issues (my Spanish is passable but not nearly as fluent as a native speaker). Note that the Call for Papers is open until the end of August.

Since you might be hesitant to consider going to Cuba from the United States, I wanted to share with you how it works.

First, tourism to Cuba for Americans is still illegal. However, the State Department has come up with a list of 12 categories which qualify for visiting.

12 Visa Categories for Cuba

In the case of CubaConf, you will choose either number four “Professional research and professional meetings” or number five “Educational activities”. I guess number six might work “Public performances, clinics, workshops, athletic and other competitions, and exhibitions” since it is kind of a workshop, but I’d stick with the first two. Since I am a free software professional, I plan to use number four, as I consider this a professional meeting.

Note that Cuba could care less about why you are there – this is a requirement of the US government.

Second, once you have a legal travel category, you’ll need a visa. In speaking with my favorite airline, American Airlines (they offer direct flights to Havana from Charlotte, NC, and Miami, FL), once you book your travel they will outsource the visa process to Cuba Travel Services who will handle the whole thing via e-mail. The visa costs $50 and it looks like there may be a $35 fee, but I’m not sure if the fee applies if you are referred via the airline and it may be built into the price of the ticket.

Speaking of things included in the price, the third thing to consider is that all Americans traveling to Cuba must have non-US health insurance. This is included as a $25 charge when you purchase your ticket.

That covers much of the “getting there” part. The fourth, and in my mind most important thing to know is that Cuba is still very much a cash-only country. American banks are still not doing business there so your credit cards won’t work, nor will the ATM, so you’ll need to bring cash. I verified this with calls to Bank of America, Chase and Citi – currently none of those banks have cards that work in that country.

There are two types of currency in use: The Cuban National Peso (CUP) and the Cuban Convertible Peso (CUC) or “kook”. The CUC is pegged to the dollar and is the currency used by most visitors. Luckily, Havana is a pretty safe place, although I still won’t want to carry around a lot of money if I can avoid it.

I’m not sure where I will stay. Being a big Marriott fan I do have the option to stay at the Four Points Sheraton, but it seems to be pretty far away from the Colegio Universitario San Gerónimo where the conference will be held. Most people visiting stay in a “casa particular” which is a room in someone’s house, and it appears that Airbnb is also in Cuba.

I plan to use the open source way and just ask my friends organizing the conference where I should stay. It is very easy to do, as they have set up a Telegram channel for the conference. While Spanish is the main language in the channel, English is welcome, and if you are thinking about coming to CubaConf I would consider going there first.

I am very exited about the opportunity to visit Havana in November. Despite the modern history between the US and Cuba, I know I’ll make some new friends.

Software libre crea amistades inmediatas.

Horizon™ Version 20 Released

Just a heads up that version 20 of Horizon has been released.

Since version 20 coincides with the 20th anniversary of the film The Fifth Element, we’ve decided to use characters from that movie as codenames for this release. Version 20.0.0 is called “Leeloo”.

This release continues our commitment to rapid releases in the Horizon product line, and is mainly focused on bug fixes, small enhancements and code cleanup. We have removed all use of Castor for the parsing of XML files and replaced it with JAXB, and number of deprecated events have been removed from the system.

Probably the biggest new feature is a topology provider that can be used to create custom maps. The Asset Topology Provider generates a GraphML topology based on node metadata including asset fields.

You can read the announcement and for more information, check out the release notes.