The next to the last day of Dev-Jam was pretty much like the one before it, except now it was quite clear that Dev-Jam was coming to a close (sniff).

I actually managed to get some of the work done that I wanted to do this week, namely to start working on the next version of my OpenNMS 101 video series. A lot changed in Horizon 24 and now the videos are a little off (especially when it comes to alarms) and I want to fix that soon.

I did make one bad decision when I purchased take-away sushi from the Union, but I was lucky that I got over it quickly (grin)

It’s so nice to be able to break out into little groups and share what is going on in OpenNMS. Jesse gave an in-depth talk on ALEC (and I’ll be presenting it at this year’s All Things Open conference).

It wasn’t all work, though.

A group of people had gone to the Mall of America on Sunday, and Markus bought a Rick and Morty card game that seemed pretty popular. Parasites!

For dinner I ordered some delicious pizza from Punch as many people wanted to stay in and finish up their projects in time for tomorrow’s “Show and Tell”.

It’s hard to believe Dev-Jam is almost over.

Not much to add on Day 3 of Dev-Jam. By now the group has settled into a routine and there’s lots of hacking on OpenNMS.

As part of my role as “cruise director” Mike and I ran out for more snacks.

On the way we stopped by the Science Museum of Minnesota to pick up a hoodie for Dustin. As fans of Stranger Things we thought we should get our Dustin the same hoodie worn by Dustin in the show. The one in the show was apparently an actual hoodie sold by the museum in the 1980s, but it was so popular they brought it back.

While not exactly the “Upside Down” in the evening the gang descended on Up-Down, a barcade located a few miles away. Jessica organized the trip and folks seemed to have a great time.

The combination bar and arcade features vintage video games

as well as pinball machines

Of course, there was also a bar

Good times.

While the OpenNMS team does a pretty good job working remotely, it is so nice to be able to work together on occasion. Here is an example.

I wanted to explore the current status of the OpenNMS Selenium monitor. My conclusion was that while this monitor can probably be made to work, it needs to be deprecated and probably shouldn’t be used.

I started off on the wiki page, and when I didn’t really understand it I just looked at the page’s history. I saw that it was last updated in 2017 by Marcel, and Marcel happened to be just across the room from me. After talking to him for awhile, I understood things much better and then made the decision to deprecate it.

The idea was that one could take the Selenium IDE, record a session and then export that to a JUnit test. Then that output would be minimally modified and added to OpenNMS so that it could periodically run the test.

The main issue is that the raw Selenium test *requires* Firefox, and Firefox requires an entire graphics stack, i.e. Xorg. Most servers don’t have that for a number of good reasons, and if you are trying to run Selenium tests on a large number of sites the memory resources could become prohibitive.

An attempt to address this was made using PhantomJS, another Javascript library that did not require a graphical interface. Unfortunately, it is no longer being maintained since March of 2018.

We’ve made a note of this with an internal OpenNMS issue. Moving forward the option looks like to use “headless Chrome” but neither OpenNMS nor Selenium support that at the moment.

We still have the Page Sequence Monitor. This is very efficient but can be difficult to set up.

Playing with that took up most of my morning. It was hard staying inside because it was a beautiful day in Minneapolis.

Most of my afternoon was spent working with OpenNMS customers (work doesn’t stop just because it is Dev-Jam) but I did wander around to see what other folks were doing.

Jesse was playing with a VR headset. The OpenNMS AI/Machine Learning module ALEC can create a visualization of the network, and he wrote a tool that lets you move through it in virtual reality (along with other people using other VR headsets). Not sure how useful it would be on a day to day basis, but it is pretty cool.

That evening most of us walked down the street to a pretty amazing Chinese restaurant. I always like bonding over food and we had discovered this place last year and were eager to return. I think the “bonding” continued after the meal at a bar across the street, but I ended up calling it a day.

Dev-Jam officially got started Monday morning at 10:00.

I usually kick off the week with a welcome and some housekeeping information, and then I turn it over to Jesse White, our project CTO. We do a roundtable introduction and then folks break off into groups and start working on the projects they find interesting.

This year we did something a little different. The development team scheduled a series of talks about the various things that have been added since the last Dev-Jam, and I spent most of the day listening to them and learning a lot of details about the amazing platform that is OpenNMS. While we had some technical difficulties, most of these presentations were recorded and I’ll add links to the videos once they are available.

Jesse started with an overview of the main development projects over the last year. Sentinel is a project to use the Kafka messaging bus to distribute OpenNMS functionality over multiple instances. While only implemented for telemetry data at the moment (flows and metrics) the goal is to enable the ability to distribute all of the functionality, such as service assurance polling and data collection, across multiple machines for virtually unlimited scalability.

After the Sentinel work, focus was on both the OpenNMS Integration API (OIA) and the Architecture for Learning Enabled Correlation (ALEC).

The OIA is a Java API to make it easier to add functionality to OpenNMS. While it is used internally, the goal is to make it easier for third parties to integrate with the platform. ALEC is a framework for adding AI and machine learning functions to OpenNMS. It currently supports two methods for the correlation of alarms into situations: DBScan and TensorFlow, but is designed to allow for others to be added.

The current development focus is on the next version of Drift. Drift is the feature that does flow collection, and there are a number of improvements being worked on for “version 2”.

Markus von Rüden gave the next talk on contributing to OpenNMS. He covered a number of topics including dealing with our git repository, pull requests, test driven development and our continuous integration systems.

Matt Brooks presented an overview on how to leverage Karaf to add functionality to OpenNMS. Karaf is the OSGi container used by OpenNMS to manage features, and Matt used a simple example to show the process for adding functionality to the platform.

Extending on this was a talk by Chandra Gorantla about using the OIA with an example of creating a trouble ticketing integration. OpenNMS has had a ticketing API for some time but this talk leveraged the improvements added by the new API to make the process easier.

Following this was a talk by David Smith on ALEC. He demonstrated how to add a simple time-based correlation to OpenNMS which covered a lot of the different pieces implemented by the architecture, including things like feedback.

That ended the development overview part of the presentation but there were two more talks on Docker and Kubernetes.

Ronny Trommer gave a short overview of running OpenNMS in Docker, covering a lot of information about how to deal with the non-immutable (mutable?) aspects of the platform such as configuration.

This was followed by an in-depth talk by Alejandro Galue on Kubernetes, running OpenNMS using Kubernetes and how OpenNMS can be used to monitor services running in Kubernetes. While Prometheus is the main application people implement for monitoring Kubernetes, it is very temporal and OpenNMS can augment a lot of that information, especially at the services level.

These presentations took up most of the day. Since it is hard to find places where 30 people can eat together, we have a tradition of getting catering from Brasa, and we did that for Monday night’s meal.

Jessica Hustace, who did the majority of the planning for Dev-Jam, handed out this year’s main swag gift: OpenNMS jackets.

Yup, I make this look good.

For the fourteenth time in fifteen years, a group of core community members and power users are getting together for our annual OpenNMS Developers Conference: Dev-Jam.

This is one of my favorite times of the year, probably second only to Thanksgiving. While we do a good job of working as a distributed team, there is nothing like getting together face-to-face once in awhile.

We’ve tried a number of venues including my house, Georgia Tech and Concordia University in Montréal, but we keep coming back to Yudof Hall on the University of Minnesota campus in Minneapolis. It just works out really well for us and after coming here so many times the whole process is pretty comfortable.

My role in Dev-Jam is pretty much just the “cruise director”. As is normal, other people do all the heavy lifting. I did go on a food and drink run which included getting “Hello Kitty” seaweed snacks.

Yudof Hall is a dorm. The rooms are pretty nice for dorm rooms and include a small refrigerator, two burner stove, furniture and a sink. You share a bathroom with one other person from the conference. On the ground floor there is a large room called the Club Room. On one side is a kitchen with tables and chairs. On the other side is a large TV/monitor and couches, and in the middle we set up tables. There is a large brick patio that overlooks the Mississippi River.

The network access tends to be stellar, and with the Student Union just across the street people can easily take a break to get food.

We tend to eat dinner as a group, and traditionally the kickoff meal is held at Town Hall Brewery across the river.

It was a pretty rainy day but it stopped enough for most of us to walk over the bridge to the restaurant. You could feel the excitement for the week start to build as old friends reunited and new friends were made.

When we were setting up the Club Room tables, we found a whiteboard which is sure to be useful. I liked the fact that someone had written “Welcome Home” on it. Although I don’t live here, getting together with these people sure feels coming home.