30 Years

I missed noting it, but 30 years ago on Saturday, June 9th, 1984 I graduated in the third graduating class of the North Carolina School of Science and Mathematics. It was an amazing two years and I’m happy to say that I’m still in touch with many of the people I met there.

The speaker at my graduation was Ron McNair, the first African-American astronaut. Earlier that year, in February, I got to see him launched into space on Space Shuttle mission STS-41-B. Unfortunately, he was one of the seven people killed in the Challenger disaster a couple of year later.

While most people, I would assume, don’t remember the speech given at their high school graduation, I remember him talking about daring to be “a flea weight in a heavy weight world”. I took those words to heart and it is one of the reasons OpenNMS is able to challenge products from the likes of HP and IBM.

I should note that on Sunday, June 10th, 1984, I started working as a machine operator in a plastics injection molding plant in Asheboro, NC. In retrospect, it wouldn’t have hurt me to take a day off but I guess that just how we open source geeks roll.


2014 Dev Jam – Day 6

Friday was pretty much the “Dev Jam Results Show”. Mike suggested that the various teams get together and present their work. He recorded it live from his iPhone using Ustream and the raw (and I mean raw) video is available for your viewing pleasure.

The first stream covered the following topics.

Ben, Ron and Matt R. – AngularJS based webUI

That OpenNMS could use a new user interface is a given, and the decision was made to base it on a technology called AngularJS. The first demo shows off some of the work that was done to build a framework for the new GUI.

This includes a “plugin” architecture that will make it easy to add functionality to the system as well as to embed existing code that, while not written in Angular, is still useful. Ron even managed to get KSC reports to run under the new code, complete with dynamic updates.

Work that still remains to be done include formal authentication (currently the new GUI just gets a session cookie from the old one) as well as a greater granularity for ReST permissions, as now normal users get a lot of data and admin users get all the data. This would be very useful for things like multi-tenancy.

Alejandro – Requistion Manager

When we wrote the provisioner, we knew we had something special as no other management system seems to take discovery as seriously as OpenNMS. As more and more people find novel ways of using this system, we realized that the user interface could use some improvement. In this section Alejandro demonstrates the changes he has made to the interface for creating and managing requisitions, also built on Angular.

Craig Gallen – High Frequency Trader GUI

Those of you that follow OpenNMS in the news might have seen a press release a couple of weeks ago from a financial trading services company called TMX Atrium Networks. Dr. Gallen, our man in the UK, worked with them to build an interface for monitoring latency across the network.

Matt and Eric – newts

Yesterday I talked about the New Time Series database that we are building as the data storage backend for OpenNMS. I lifted a lot of that from this portion of the demonstrations. The ability to have an incredibly fast and highly scalable data store is key for our goal of making OpenNMS the de facto network management platform of choice.

The second video stream features a talk about “snee-po”

Seth – SNMnepO

SNMnepO, or OpenNMS spelled backwards, is a project to create a distributed data collector with horizontal scalability. The idea is to add data collection to our remote poller, and Seth’s demonstration shows data collection being performed with the collectd process disabled.

Coupled with the newts data storage backend, this new distributed collector will insure that OpenNMS can scale to meet any data collection needs in the future.

The final stream focuses on work being done by our German team.

Christian – Outage Timeline

Christian demonstrates the new outage timeline that I talked about earlier in the week.

Dustin – RRDtool export via ReST

Dustin shows a new feature that exposes collected data from the RRD files via ReST. This can allow for another integration point where collected data can easily be used by other applications.

Ronny – PRIS

The last demo was done by Ronny (presenting work that was also done by Dustin) on the Provisioning Integration Server (PRIS). As mentioned above, the ability to tightly integrate OpenNMS with provisioning systems is a key feature of the platform. Originally done to integrate with OCS Inventory NG, the system has been extended to allow for integration with pretty much any system.

Considering that these demonstrations were pretty much ad hoc, I was delighted to see how much was accomplished in just a week. It is one of the main reasons I look forward to Dev Jam every year.

We celebrated that evening with a trip to Republic.

Let’s just say the evening went a little downhill from there, but I did manage to make it back to the dorm.

2014 Dev Jam – Day 5

Thursday turned out to be picture day. We had some people leaving a little early so we decided to get our group picture done in the morning. Recently the school added a new Goldy Gopher statue near our dorm, so it was the logical place for a photo.

It also worked out that Goldy would fit in a 3XL OpenNMS shirt (grin)

I even took the opportunity for one of them there “selfie” thingies:

Don’t expect to see many more, but I was told it would be “ironic” (grin).

There was some real work done as well. We are getting much closer to a 1.0 release of newts (http://www.newts.io), the NEW Time Series database built on Cassandra. The speed is pretty amazing, with sustained writes of 50K+ data points per second.

For testing we’ve been using some weather data that contains 1.2 billion data points, but even at 50K per second it takes six hours to import.

Note that this was done on Matt’s laptop and one Cassandra node. On server hardware it should be much faster, and Matt and Eric have worked very hard to make it linearly scalable: two nodes are twice as fast, four nodes are four times as fast, etc.

The whole Internet of Things paradigm requires the ability to manage massive amounts of time series data and we are getting close to making it a reality.

I am also dealing with the reality that I ate way too much pizza in the last 24 hours. Thanks to Chris Rodman and the good people at Papa John’s Pizza, we had a pizza feast:


Order of the Blue Polo Update

It has been awhile since we had an entry into the Order of the Blue Polo, so I thought it would be cool to blog about it.

While the Order of the Green Polo (OGP) is the governing body of the OpenNMS Project, we wanted to find a way to recognize users who didn’t quite have the time necessary to dedicate to the project for OGP membership. It also solves a problem for us: we have lots of amazing users of OpenNMS, but we can’t always talk about them.

Membership in the Order of the Blue Polo is pretty straightforward. Simply send us an e-mail on why you like OpenNMS, preferably with a quick list of the number of devices/interfaces/services you are monitoring. If we can publish it on the wiki along with your name and your company’s name, we’ll send you a limited edition blue OpenNMS polo. So please, no gmail.com or yahoo.com e-mails – we really need to be able to verify your company.

This helps us because like attracts like, and perhaps someone will read about how you are using OpenNMS and decide that it fits in with their needs.

The latest entry into the Order comes from Paul Cole, a contractor and Environment Canada. He writes:

I am a contractor working currently for Environment Canada (under Shared services Canada).

It was a beautiful thing to be given the OpenNMS project in order to map out and bring together teams to monitor and work on the very large LAN. A multitude of tools and scripts were used custom to each area, and we are now moving towards unification.

The visibility and baseline abilities of OpenNMS are fantastic, and the new topo-mapping/geo-mapping features are looking fantastic come version 1.14!

Our network size is larger than the current scope of nodes we are testing with, but OpenNMS is managing it pretty smoothly and seamlessly.

I have released some of the scripts I wrote to contribute back to the community that are based on version 1.10-1.12 and hope they help more people realise the power and scalability of the product.

Currently our OpenNMS build is monitoring over 15k nodes and 20k interfaces and 25k or more services (exact numbers can be extracted but it grows every quarter), on an 8 core server with 16GB of RAM using the discovery method.

It is keyed to discover every IP and node it can, and monitor switches/UPSs and routers and select key devices for management, to send email alerts to the appropriate regional teams when a device is down or a specific threshold or alert is received.

I can’t of course send out network diagrams , but I can send a screenshot of the geo-map to give an idea of how it goes.

Anarchy OpenNMS in the UK

OpenNMS has a strong presence in both Europe and the UK, and much of the UK effort is driven by Dr. Craig Gallen.

He has created a new website and newsletter aimed at OpenNMS users in the United Kingdom and Ireland (but, of course, it is open to anyone).

The new website can be found at opennms.co.uk and I think it is pretty spiffy (“spiffy” is a proper English word, correct?). There is also an occasional newletter list focusing on OpenNMS events in the region, so if you are interested in such things please register.

The first big push to raise awareness of OpenNMS as well as provide training is a series of OpenNMS workshops to be held around the area. In Craigs words:

Don’t just expect to be lectured to. This will be a participative event. These workshops will stretch your understanding of Operational Support systems and help you to begin thinking through how you can adapt OpenNMS to address some of the key problems in Network and Service Management.

London – Monday 30 June 2014

Location: University of London Union, Malet Street, London, WC1E 7HY

Birmingham – Tuesday 1 July 2014

Location: IET Aston Court, 80 Cambridge Street, Birmingham, B1 2NP

Rochdale (near Manchester) – Wednesday 2 July 2014

Location: Zen Interent Ltd. Sandbrook Park, Sandbrook Way, Rochdale, OL11 1RY

Glasgow – Friday 4 July 2014

Location: IET Glasgow: Teacher Building, 14 St Enoch Square, Glasgow, G1 4DB

There is a cost associated with the workshops, but there are a number of discounts available. There is an early bird discount of 10% if you book before 13 June, and if you are a current commercial support customer or a non-profit there is a further reduction in cost. Also, sending more than one person creates even more discounts.

So if you are a non-profit, buy a commercial support contract and then book a whole bunch of people before 13 June and you’ll be saving money hand over fist (grin).

Visit the Registration Page for more details.

This is a wondeful way to get up to speed on OpenNMS and I appreciate the effort Craig put into making these workshops available.

2014 Dev Jam – Day 4

First let me interrupt this blog post with a special announcement. A rather onerous security bug was discovered in OpenNMS that would allow any authenticated user to access pretty much any file on the system.

We felt it was bad enough to actually create a fix in the 1.10 branch as well as in the current stable, 1.12, so please consider upgrading at the earliest possibility.

Hat tip to Martin Laercher for reporting it.

Wednesday marked the halfway point in the week, and everyone seems to be in a good groove. With everyone able to work together in person, a lot of nifty things are getting done, including an upgrade to the latest version of Drools Expert.

The integration of OpenNMS with Drools allows for very powerful alarm correlation, and by migrating to Drools 6.0.1 it just got more powerful.

Wednesday also marked the day of the Twins game. For the past two years we’ve taken everyone to the Twins ballpark to watch a major league baseball game. It’s a beatiful place for baseball:

although they usually stuck us in far right field. Also, for the last two games the Twins played the Royals, and lost both times.

This year they put us in far left field:

and the Twins faced the Brewers. Since Milwaukee is close to Minneapolis there were a lot of Brewers fans in the stands, but the home team pulled it out for the win.

We got our name on the big board, too, which was cool, and Jeff was quick enough to catch a picture.

2014 Dev Jam – Day 3

By Tuesday things are pretty much underway. People have divided up into teams and are busy working on various improvements to OpenNMS.

All of this is managed by Bamboo, our build and test system. We have a dashboard view of the status of each branch: green means good and red is bad.

Last year DJ was working on a Raspberry Pi controlled stoplight, and it is now complete and tied into Bamboo. Again, green means good and red means bad, but it also pulses when building.

Tuesday evening we decided to head to Mall of America for some blatant consumerism (and dinner). I hired a school bus to drive us there, and while a “short bus” might have been more appropriate for this crowd, we had too many people to fit.

As part of the evening’s festivities, we attended a Star Trek exhibition at the Mall.

It contained a number of costumes and props from the various Star Trek television series and movies. It wasn’t too amazing but it was fun, and folks seemed to enjoy it.

Trust and the Internet of Things

One of the main things we are focusing on at Dev Jam is scalability. The goal of OpenNMS has always been to become the de facto management platform of choice for everyone, and as more and more things become connected to the Internet, scalability will be a huge issue. It’s one thing to monitor a drink machine and quite a different thing to monitor millions of them.

The main performance bottleneck in OpenNMS has always been the storage of time series data. While much of it can be addressed via hardware, we realized we needed a write-optimized system that could perform at scale. The result of that work, lead by Eric Evans, is NewTS, the New Time Series database built on Cassandra.

As a left leaning libertarian, I’m really concerned about the privacy issues of the Internet of Things. One of the reasons I work on OpenNMS is to insure that the best platform for managing this data is also one that can be privately owned. It will be the end user’s choice one how that data is stored and shared.

Yesterday Apple announced a number of new initiatives at their WWDC keynote. These included iHome, a home automation platform, and iHealth, a platform for gathering personal biometric data. Among the new shiny was a total lack of concern about privacy. An example given was an integration between iHome and Siri, where you could tell Siri “I’m going to bed” and certain actions would happen. What people tend to forget is that almost all of Siri’s processing is done on Apple servers, and if you tell Siri you are going to bed, you are also telling Apple and who knows who else. The potential for 1984-like abuse is there, and it will be up to Apple users to either trust Apple or do without.

But does this have to be the case? Do users have to give up privacy or blindly trust in third parties in order to access useful technology?

I don’t think so.

The technology is there, and I believe eventually society will demand it.

One of the oldest examples of such technology is public key encryption. It’s a beautiful system, and while the encryption aspect is important, so is the ability to digitally “sign” things. What’s brilliant about it is that no personal information has to be given up – if one trusts the key then one can trust the signature even without knowing who signed it, and third parties can verify the signature as well.

I am eagerly awaiting the release of the Angel open source health sensor. I have a strong interest in tracking my personal health data, but I also don’t want to share it with a third party unless I choose to do so. None of the popular sensors, to my knowledge, have the option of keeping that data private if you want it to be useful.

So I’m constantly on the lookout for companies and products that “get it”, that understand my desire to only share the information I choose. I want to be marketed to, show me cool things I’m interested in, but on my terms.

One company I recently came across is called Personagraph. On the surface it appears to be another analytics company, but if you dig deeper you can find that they have a strong interest in personal privacy. I learned about them on one of my recent trips to Silicon Valley, but I had to dig to find references that reflect what I was told.

Of their three main products, I’m really drawn to PG Protect. The idea is as follows:

Let’s say that I have an application that collects several hundred metrics about me. For example, totally as a mind experiment, assume that I like to watch The Big Bang Theory in the nude while eating popsicles. I might not want to share the nudity aspect with anyone, but I am okay with letting people know I like the show and that I like popsicles. In a traditional marketing application, I would need to reveal these things to a third party, which would in turn sell them to interested companies.

But what if instead of individual metrics, some sort of aggregate score was created from my likes and interests, and that anonymous score was the only thing that was presented to the third party? They might be looking for people who score high on a number of metrics and yet they could be matched with me without knowing exactly which metrics I trigger. I could then be presented with offers and ads, and then the choice could be up to me if I want to engage. Speaking of “engage”, a product like Personagraph’s PG Engage could be used by the company doing the marketing to see how well their campaign was doing, without having to know any personal information about me.

Pretty cool, huh? What I like about their products is that they provide trust without having you to take it on faith.

Unfortunately, it’s hard to get the privacy angle from their website. I did find a little YouTube ad that touches on it:

but the real meat can be found in this white paper. If you care about privacy, I encourage you to check it out as well as to make sure the companies providing the products and services you use are aware that privacy is important to you.

2014 Dev Jam – Day 2

The conference kicked off Monday morning. I did a general introduction and then handed it over to Matt (Brozowski), who in true un-conference fashion turned it over to the group.

One perennial topic with respect to OpenNMS has been to improve the GUI. One technology we are considering is AngularJS, so after the initial session a group of guys went across campus to one of the tech center classrooms so that Matt (Raykowski) could do a presentation on it.

But not all of the great things that come out of Dev Jam have to be on such a grand scale as “a new GUI”. Markus showed me a very useful feature called “Outage Timelines” that is already complete:

It allows to you quickly see the impact of outages in the last 24 hours. It will definitely be in the next major release and might even make it into the next 1.12 version.

For dinner Monday night I had Brasa cater once again. Even though I backed off on the size of the order, we tend to have a lot of leftovers so I figured it was best to have it early in the week. It was amazing as usual.

After dinner I had to walk some of it off, so I wandered around campus. The sky was threatening to storm, and the sunset against the Weisman Art Museum was beautiful.

So far the vibe at this years conference has been even more positive than usual. This is a week that reminds me why we work so hard the rest of the year. While OpenNMS isn’t a huge project, the people involved, myself exempted, are giants. It’s great to be able to spend time with them, no matter how brief.

2014 Dev Jam – Day 1

Sunday marks the official start of Dev Jam, as people begin to arrive for the conference. This year we have 26 people from four countries (since Alejandro now works for us in the US, I’m not counting Venezuela).

As the conference grows it is becoming harder and harder to fit it into everyone’s schedule, so we use that most democratic of processes: the vote. Once the University gets back to us with available dates, we have people vote on both their preferred week and which other weeks they could attend. We then choose the one that accommodates the most people. Unfortunately, this year the best week caused hardship for a number of long time attendees, so people like Antonio couldn’t make it. I’m hoping we find a better fit next year.

With the exception of a couple of delayed flights due to the weather (Minneapolis has recently been experiencing a lot of rain, often resulting in flooding), everyone got here safely and we gathered at the Town Hall Brewery for dinner.

Most people tried the beer (I had water, this week is pretty rough on my diet as is) but Ben discovered a “whiskey tasting flight” that he said was both good and a good value. All in all, it was a nice way to kick off the week.