2015 Dev-Jam: Day Five

Sorry for the week delay on this post, but this happened.

The last day of Dev-Jam is always bittersweet for me. I’m sad that it is over, but I also get to see all the wonderful “new shiny” people have been working on. Friday we do demos.

This year we made an attempt to record each demo. Just click on the picture to see it on the YooToobz. Videos, yay!

First up was Ben. Ben is the architect of our new mobile app, OpenNMS Compass. Available for both iOS and Android, it may even turn into our next overall user interface. To do that, it needs graphs, so Ben demonstrated how you can now display graphs in Compass. You can even set “favorites” so they show up on your main screen.

Dev-Jam Demos: Ben

Markus von Rüden spent the week working on something fun: digitizing our mascot and kiwi overlord, Ulf. He demonstrated this work in a game. While it wasn’t completely finished, when Ulf died would he split in half to reveal a kiwi (fruit) center. Cute. Unfortunately, no video and no wiki page (yet).

Dev-Jam Demos: MvR

Christian presented a new way to represent issues within OpenNMS, a “heat map“. It works with both alarms and outages.

Dev-Jam Demos: Christian

Jesse presented something that literally gave me goosebumps. Using our new integration with Newts, you can search for similar data within OpenNMS. So if there is say, a spike, you can search through all the other metrics to see if there are other data sources that spike at the same time.

Dev-Jam Demos: Jesse

David S. presented a new northbound interface for sending alarms to other systems via JMS. He used ActiveMQ as a proof of concept.

Dev-Jam Demos: David S.

Ron created a couple of new features. The first was the ability to see polls as events, including how long each poll took (if available). He also added the ability to create a consistent color scheme across performance graphs.

Dev-Jam Demos: Ron

Umberto created another real exciting feature – the ability to export in real time OpenNMS events to Elasticsearch. Since OpenNMS can handle thousands of events a second, sending them to a system built to analyze such data could be very useful. Umberto was sponsored to attend by the OpenNMS Foundation.

Dev-Jam Demos: Umberto

A second OpenNMS Foundation attendee was Marcel. He worked on improving data collection for Fortinet devices.

Dev-Jam Demos: Marcel

Another cool feature was Dustin’s custom data collection script tool. Sometimes OpenNMS gets criticized for not using SSH to collect data and perform montoring. The reason it doesn’t is rather simple: it’s a stupid idea. It usually requires that you set up keys with null passphrases, and often they connect as root. Despite the security issues, it is also a resource hog and can’t scale. We have always recommended using an extensible SNMP agent like Net-SNMP, but it can be some effort to set up.

Dustin’s feature allows you to put collection scripts in a special folder on the server, and OpenNMS will automatically collect the data. All you need to do then is to add a graph definition and you’re done.

Dev-Jam Demos: Dustin

Ronny discussed running OpenNMS in Vagrant and Docker containers. Neato.

Dev-Jam Demos: Ronny

DJ was frustrated in how long it can take to compile OpenNMS if tests are enabled. OpenNMS is heavily instrumented with software tests. These can be broken into “unit” tests and “integration” tests. Maven (the build system used by OpenNMS) can be configured to separate them. Since unit tests should be small and quick, they can be run every time with the integration tests only run for regression.

Dev-Jam Demos: DJ

Finally, Seth presented the work he did this week which was focused on changes needed to support OpenNMS Minion. Minions are little, stand alone processes that can perform basic monitoring and data collection, which is then forwarded up to an OpenNMS Dominion instance.

Dev-Jam Demos: Seth

While the last Dev-Jam always seems to be the best Dev-Jam, I think it was true this year. This work will go along way toward positioning OpenNMS for the coming Internet of Things, and as always it is amazing to see what brilliant people can do when given the opportunity to work together.

We ended the day at Surly Brewing Company. The beer was delicious and the company stimulating. I only got one non-blurry picture, and unfortunately my pants fell down when I stood up to take it.

Dev-Jam: Surly Brewing

Sorry.

I hope to see everyone at the OpenNMS Users Conference in September. I promise to keep my pants on.

2015 Dev-Jam: Day Three

It’s hard to believe that this year’s Dev Jam is half over. It seems to take forever to get here and then it is over too soon.

Lots of cool stuff going on. David Schlenk has written a Java Message Service (JMS) northbounder interface. While targeted at Apache ActiveMQ, it should work with any JMS provider, and it is another great tool to have for the OpenNMS platform.

Christian did a cool little hack using a Blink(1) USB light. If you have a Blink(1) you can plug it into your laptop and then run a little Java app. The app will connect to your OpenNMS instance and then the color will change based on the severity of alarms. Cool.

Dev-Jam Key Blink(1)

I also participated in my first GPG key signing. Jeff organized it to increase our “web of trust” and once I got into it, it was kind of fun.

Dev-Jam Key Signing

He promised cake, and for once the cake wasn’t a lie:

Dev-Jam Key Signing Cake

The cake went well with dinner, which came from the always amazing Brasa:

Dev-Jam Key Brasa

Most of us ate out on the deck. This is the view looking out toward the Mississippi:

Dev-Jam View

While we have held Dev-Jam in locations other than UMN, it has become a lot harder to do so since we get treated so well here. While most attendees have been to previous Dev-Jam events, we always have a few new people, and many of them end up staying off campus in a nearby hotel. There is something about staying in a dorm room that bothers them – perhaps it was a bad experience in college.

So I thought it would be a public service to actually show the rooms we get at Yudof Hall.

Each person at Dev-Jam gets their own room. While these rooms tend to hold at least two students when classes are in session, during the summer they are singles. You get a sink and a little kitchen with a microwave, two burner stove and refrigerator.

Dev-Jam Dorm Kitchen

There is a single bed, desk, an armoire (closet) and a chest of drawers.

Dev-Jam Dorm Bed

You even get to control the temperature in the room. I like mine on the cool side and the air conditioning works quite well.

Each room shares a bathroom with a toilet and a shower with one other person from Dev-Jam. While these rooms might be a little close with two people, they are perfect for one. Plus they are incredibly convenient since the event is held in the downstairs Club Room.

So, if you ever decide to come to Dev-Jam, don’t hesitate to stay on campus.

2015 Dev-Jam: Day One

Ah, it’s that most wonderful time of the year: Dev Jam. Once again we have gathered at Yudof Hall on the University of Minnesota campus.

Dev-Jam OpenNMS Sign

This year marks the tenth time we’ve gathered together to spend a week hacking on OpenNMS. The first one was held in 2005 at my house, and every year since then (with the exception of 2009) we’ve managed to have another one. I’m being sincere when I say that I look forward to this week almost more than any other.

Plus, I get to wear my “special” Dev-Jam T-shirt:

Kiwis come from T-rex

Our project’s mascot, Ulf, showed up at a Dev-Jam many years ago as a gift from Craig Miskell (who came from New Zealand). That he became our mascot wasn’t intentional, but then again it seems in open source all that is required is to create an environment conducive to great things and great things will happen. This year we have people from the US, Canada, Germany, Italy and the UK, all working to make OpenNMS even more awesome.

I wore that shirt as I kicked off this year’s conference. I also got to announce that Dustin Frisch and Jesse White have been inducted into the Order of the Green Polo. It is important to note that both Dustin and Jesse worked with us in the Google Summer of Code project and now are key members of our team. I also got to formally announce that Jesse is now the Chief Technical Officer of The OpenNMS Group. I then turned the meeting over to Seth Leger (our VP of Engineering) and Jesse as Dev-Jam is more about coding than me running my mouth. My main role is to serve as “Julie” the cruise director.

Dev-Jam gives us, as a community, a chance to work:

Dev-Jam work area

and a chance to share:

Dev-Jam Presentation

Here, Umberto talks about his work integrating OpenNMS with Elasticsearch and Kibana (code available on GitHub).

We also get to play. Here, MvR and Jessica are working on modeling Ulf in 3D:

Dev-Jam MvR and Jessica

and DJ and Mike Huot (my co-cruise director) play with a 3D printer:

Dev-Jam 3D Printer

The first day of Dev-Jam seemed to fly by this year. Now that light rail is available from UMN we can travel more easily about the city. In the evening, some of us went to Mall of America while I and others saw Mad Max: Fury Road (which I highly recommend).

Day Two? Minnesota Twins, baby.

Dev-Jam 2015 – Magical Number 10

We are just about a month away from one of my favorite weeks of the year: The OpenNMS Developer’s Jamboree, or Dev-Jam.

This is the tenth one we’ve had, which is hard for me to believe. I think it is a testament to the community around the OpenNMS Project that we can have these year after year (and not a testament to the fact that I’m quickly becoming an “old guy”).

We have people from all over the world who contribute to OpenNMS, and for one week out of the year we get together to hack and hang out. It was an “unconference” before such things were popular.

The first one was held at the Pittsboro OpenNMS HQ in 2005, but we quickly learned that we needed a bigger venue. The requirements for a successful Dev-Jam are as follows:

  • A room big enough to hold everyone
  • Fast Internet
  • A place for everyone to sleep
  • Food

We found a great home for Dev Jam at the University of Minnesota’s Twin Cities campus in Minneapolis, specifically in a dorm called Yudof Hall. We lease the downstairs “club room” which is a large rectangular room that is big enough for our crowd. On one side is a kitchen and on the other side is an area with a television and couches. In the middle we set up tables for everyone to work.

We also get rooms in the same dorm, so people can come and go as they please. Some people like to get up in the morning. Others stay up late and don’t come down until noon. The campus offers a number of places to eat, and in the evening we can walk to a restaurant for dinner and drinks. We try to see a Twins game while we are there as well as take a trip to Mall of America.

This will be the first year that access to the light rail system is available from campus, which will make getting around so much easier.

For those of you who haven’t spend a lot of time embedded with an open source project, you probably don’t understand how much fun an event like this can be, or why just writing about it makes me eager for June to arrive. Technically I’ll be at work, but it is unlike any other job I’ve ever had.

If you would like to come, we still have a few places left. Check out the Registration page for more information. Everyone is welcome, but be advised that this is a “code” heavy conference with little formal structure. For more casual OpenNMS users, there is the User’s Conference in September.

Hope to see you at Dev-Jam, and if not there, at the OUCE.

2014 Dev Jam – Day 6

Friday was pretty much the “Dev Jam Results Show”. Mike suggested that the various teams get together and present their work. He recorded it live from his iPhone using Ustream and the raw (and I mean raw) video is available for your viewing pleasure.

The first stream covered the following topics.

Ben, Ron and Matt R. – AngularJS based webUI

That OpenNMS could use a new user interface is a given, and the decision was made to base it on a technology called AngularJS. The first demo shows off some of the work that was done to build a framework for the new GUI.

This includes a “plugin” architecture that will make it easy to add functionality to the system as well as to embed existing code that, while not written in Angular, is still useful. Ron even managed to get KSC reports to run under the new code, complete with dynamic updates.

Work that still remains to be done include formal authentication (currently the new GUI just gets a session cookie from the old one) as well as a greater granularity for ReST permissions, as now normal users get a lot of data and admin users get all the data. This would be very useful for things like multi-tenancy.

Alejandro – Requistion Manager

When we wrote the provisioner, we knew we had something special as no other management system seems to take discovery as seriously as OpenNMS. As more and more people find novel ways of using this system, we realized that the user interface could use some improvement. In this section Alejandro demonstrates the changes he has made to the interface for creating and managing requisitions, also built on Angular.

Craig Gallen – High Frequency Trader GUI

Those of you that follow OpenNMS in the news might have seen a press release a couple of weeks ago from a financial trading services company called TMX Atrium Networks. Dr. Gallen, our man in the UK, worked with them to build an interface for monitoring latency across the network.

Matt and Eric – newts

Yesterday I talked about the New Time Series database that we are building as the data storage backend for OpenNMS. I lifted a lot of that from this portion of the demonstrations. The ability to have an incredibly fast and highly scalable data store is key for our goal of making OpenNMS the de facto network management platform of choice.

The second video stream features a talk about “snee-po”

Seth – SNMnepO

SNMnepO, or OpenNMS spelled backwards, is a project to create a distributed data collector with horizontal scalability. The idea is to add data collection to our remote poller, and Seth’s demonstration shows data collection being performed with the collectd process disabled.

Coupled with the newts data storage backend, this new distributed collector will insure that OpenNMS can scale to meet any data collection needs in the future.

The final stream focuses on work being done by our German team.

Christian – Outage Timeline

Christian demonstrates the new outage timeline that I talked about earlier in the week.

Dustin – RRDtool export via ReST

Dustin shows a new feature that exposes collected data from the RRD files via ReST. This can allow for another integration point where collected data can easily be used by other applications.

Ronny – PRIS

The last demo was done by Ronny (presenting work that was also done by Dustin) on the Provisioning Integration Server (PRIS). As mentioned above, the ability to tightly integrate OpenNMS with provisioning systems is a key feature of the platform. Originally done to integrate with OCS Inventory NG, the system has been extended to allow for integration with pretty much any system.

Considering that these demonstrations were pretty much ad hoc, I was delighted to see how much was accomplished in just a week. It is one of the main reasons I look forward to Dev Jam every year.

We celebrated that evening with a trip to Republic.

Let’s just say the evening went a little downhill from there, but I did manage to make it back to the dorm.

2014 Dev Jam – Day 5

Thursday turned out to be picture day. We had some people leaving a little early so we decided to get our group picture done in the morning. Recently the school added a new Goldy Gopher statue near our dorm, so it was the logical place for a photo.

It also worked out that Goldy would fit in a 3XL OpenNMS shirt (grin)

I even took the opportunity for one of them there “selfie” thingies:

Don’t expect to see many more, but I was told it would be “ironic” (grin).

There was some real work done as well. We are getting much closer to a 1.0 release of newts (http://www.newts.io), the NEW Time Series database built on Cassandra. The speed is pretty amazing, with sustained writes of 50K+ data points per second.

For testing we’ve been using some weather data that contains 1.2 billion data points, but even at 50K per second it takes six hours to import.

Note that this was done on Matt’s laptop and one Cassandra node. On server hardware it should be much faster, and Matt and Eric have worked very hard to make it linearly scalable: two nodes are twice as fast, four nodes are four times as fast, etc.

The whole Internet of Things paradigm requires the ability to manage massive amounts of time series data and we are getting close to making it a reality.

I am also dealing with the reality that I ate way too much pizza in the last 24 hours. Thanks to Chris Rodman and the good people at Papa John’s Pizza, we had a pizza feast:

(burp)

2014 Dev Jam – Day 4

First let me interrupt this blog post with a special announcement. A rather onerous security bug was discovered in OpenNMS that would allow any authenticated user to access pretty much any file on the system.

We felt it was bad enough to actually create a fix in the 1.10 branch as well as in the current stable, 1.12, so please consider upgrading at the earliest possibility.

Hat tip to Martin Laercher for reporting it.

Wednesday marked the halfway point in the week, and everyone seems to be in a good groove. With everyone able to work together in person, a lot of nifty things are getting done, including an upgrade to the latest version of Drools Expert.

The integration of OpenNMS with Drools allows for very powerful alarm correlation, and by migrating to Drools 6.0.1 it just got more powerful.

Wednesday also marked the day of the Twins game. For the past two years we’ve taken everyone to the Twins ballpark to watch a major league baseball game. It’s a beatiful place for baseball:

although they usually stuck us in far right field. Also, for the last two games the Twins played the Royals, and lost both times.

This year they put us in far left field:

and the Twins faced the Brewers. Since Milwaukee is close to Minneapolis there were a lot of Brewers fans in the stands, but the home team pulled it out for the win.

We got our name on the big board, too, which was cool, and Jeff was quick enough to catch a picture.

2014 Dev Jam – Day 3

By Tuesday things are pretty much underway. People have divided up into teams and are busy working on various improvements to OpenNMS.

All of this is managed by Bamboo, our build and test system. We have a dashboard view of the status of each branch: green means good and red is bad.

Last year DJ was working on a Raspberry Pi controlled stoplight, and it is now complete and tied into Bamboo. Again, green means good and red means bad, but it also pulses when building.

Tuesday evening we decided to head to Mall of America for some blatant consumerism (and dinner). I hired a school bus to drive us there, and while a “short bus” might have been more appropriate for this crowd, we had too many people to fit.

As part of the evening’s festivities, we attended a Star Trek exhibition at the Mall.

It contained a number of costumes and props from the various Star Trek television series and movies. It wasn’t too amazing but it was fun, and folks seemed to enjoy it.

2014 Dev Jam – Day 2

The conference kicked off Monday morning. I did a general introduction and then handed it over to Matt (Brozowski), who in true un-conference fashion turned it over to the group.

One perennial topic with respect to OpenNMS has been to improve the GUI. One technology we are considering is AngularJS, so after the initial session a group of guys went across campus to one of the tech center classrooms so that Matt (Raykowski) could do a presentation on it.

But not all of the great things that come out of Dev Jam have to be on such a grand scale as “a new GUI”. Markus showed me a very useful feature called “Outage Timelines” that is already complete:

It allows to you quickly see the impact of outages in the last 24 hours. It will definitely be in the next major release and might even make it into the next 1.12 version.

For dinner Monday night I had Brasa cater once again. Even though I backed off on the size of the order, we tend to have a lot of leftovers so I figured it was best to have it early in the week. It was amazing as usual.

After dinner I had to walk some of it off, so I wandered around campus. The sky was threatening to storm, and the sunset against the Weisman Art Museum was beautiful.

So far the vibe at this years conference has been even more positive than usual. This is a week that reminds me why we work so hard the rest of the year. While OpenNMS isn’t a huge project, the people involved, myself exempted, are giants. It’s great to be able to spend time with them, no matter how brief.