2015 Dev-Jam: Day Four

Since I sincerely doubt even my loyal readers get to the bottom of my long posts, I figured I’d start this one with the group picture:

Dev-Jam Group Picture

That antenna behind Goldy’s head is part of Jonathan’s project to use OpenNMS to collect and aggregate FunCube data from around the world. Can I get an “Internet of Things“? (grin)

There is this myth that just by making your software open source, thousands of qualified developers will give up their spare time to work on your project. While there are certainly projects with lots of developers, I am humbled by the fact that we have 30-40 hard core people involved with OpenNMS.

Unless you’ve gone through this, it is hard to understand. At one time, OpenNMS was pretty much me in my attic and an IRC channel. Luckily for the project that didn’t last long. My one true talent is getting amazing people to work with me. Then all I have to do is create an environment where they can be awesome.

It’s why I love Dev-Jam.

I also love pizza. Chris at Papa Johns was kind enough to send us some free pie for dinner:

Dev-Jam Pizza

Today we spent time talking about documentation. Documentation tends to be the weak point of a lot of software, and open source software in particular. The Arch Linux people do about the best job I’ve seen, but even then it is hard to keep everything current. For over a year now a group of people has been working very hard to improve the documentation for OpenNMS, and the new documentation site is most excellent.

It does take a little time to understand the navigation. The documentation is included in the source and managed on GitHub, so there is a new version for each release. But just as an example, check out the Administrator’s Guide for 16.0.2.

Written in AsciiDoc, it is now the best place for accurate information on how to use the software. We also want to extend a special thanks to the Atom project for creating the editor used to create it.

One of the things we discussed was how to deal with the wiki and the .org website. It’s not practical to duplicate the AsciiDoc information on the wiki, so the plan is to include the relevant part from the documentation in something like an iframe and use the wiki more for user stories. The “talk” page can then be used for suggestions on improving the documentation, and once those suggestions are merged they can be removed.

I had suggested that we make the wiki page the default landing page for the .org site, but Markus pointed out that we need to do a better job of marketing OpenNMS, and the landing page should be more about “why to use OpenNMS” versus “how”. I had to agree, as we need to do a better job of marketing the software. My friend Waleed pointed out in Twitter this weakness:

Twitter Comment 1 from Waleed

Twitter Comment 2 from Waleed

To better educate folks about why OpenNMS is so amazing, we are considering merging the .com and .org sites and using the .com WordPress instance for the “why you should use OpenNMS” with a very obvious link to the wiki so people can learn how to use OpenNMS. Part of me has always wanted to keep the project and commercial aspects of OpenNMS separate, but it then becomes really hard to maintain both sites.

In case you haven’t guessed, we do spend a lot of time thinking about stuff like this. (grin)

Dev-Jam Thursday

A lot of other cool stuff got done on Thursday. DJ announced that he had separated out the unit tests in OpenNMS (for features) from the integration tests (for regression). OpenNMS has nearly 7,000 junit tests (and growing). It’s the main way we insure that nothing breaks as we work to add new things to the software. But with so many tests it can take a real long time to see if your commit worked or not. This should make things easier for the developers.

It’s hard to believe that Dev-Jam is almost over. Luckily, it sets the stage for the next year’s worth of work. Since our goal is nothing less than making OpenNMS the de facto network management platform of choice, there is a lot of work to be done.

Linux on the Dell M3800

I am way behind on a number of tech reviews, but I’m hoping to catch up soon. Please bear with me.

Earlier in the year I had a disappointing experience with the new Dell XPS 13 and Linux. It was especially hard because I really loved that hardware – not since my first Powerbook have I felt such an attachment to a laptop. I was happy to learn later that there were kernel-level issues with the hardware that had to be addressed, so it wasn’t just my lack of ability in dealing with Linux.

While that story is not over, I did send it back and decided to check out Dell’s other Ubuntu offering, the powerful M3800.

I dutifully placed an order for the Ubuntu version of the laptop, and since it is much larger than the XPS 13 there were more options. I liked the fact that I could get an SSD as well as a standard HDD, so I chose the 256GB SSD option and a 1TB HDD. I travel a lot and thought it would be cool to carry more media with me while still having a fast primary drive.

The order process was pretty painless. Still not as streamlined as the Apple Store, but not too bad.

Then I waited.

My expected arrival date kept slipping. This went on for several weeks until I got an e-mail that, due to a misconfiguration, my order was canceled.


Considering that the website pretty much walks you through the ordering process and indicates any kind of impossible combination (such as a larger battery and an extra drive, since they can’t both occupy the same space) I was confused and a little torqued off.

After a few days to calm down, I decided to retry the process. It turns out that the “misconfiguration” was due to the extra drive, which was surprising. Order it with Windows? No problem. Check Ubuntu and it fails.


I did some investigation and was led to believe that the M3800 Ubuntu version ships with a vanilla 14.04 install. So I decided to pay the Microsoft tax and order the hardware I wanted, and then to base it and install Ubuntu.

This time the process was much smoother, and the laptop even arrived about a week earlier than they told me it would. It was a pleasant surprise.

The shipping box was a bit dinged up:

Dell M3800 Unboxing - Picture 1

But they did a good job of protecting the actual laptop box:

Dell M3800 Unboxing - Picture 2

There were actually two boxes, one holding the laptop and one holding accessories:

Dell M3800 Unboxing - Picture 3

All in all, it was a decent unboxing experience:

Dell M3800 Unboxing - Picture 4

The accessories included the power brick, a restore USB stick and a USB Ethernet adapter.

Dell M3800 Unboxing - Accesories

I liked having the Ethernet adapter since I’ve found installing Linux on a laptop works best when wired. While a lot of modern distros ship the proprietary wireless drivers needed, many times they aren’t enabled during install.

I got the HiDPI touchscreen option (3840×2160) and so I decided to install Linux Mint on it. I figured that if Ubuntu worked on it, I should be able to get Mint to work, and I prefer Cinnamon to Unity, plus Mint handles HiDPI screens much better than Ubuntu. Ubuntu has a scaling factor but it doesn’t really apply across the board, so you end up seeing things like clipped text under icons, etc., and sometimes the selection boxes can be very small. I believe Mint does what Apple does and just doubles everything (i.e. represents system graphics with four pixels instead of one).

This system is screamingly fast (I got the Quad Core 3.3GHz CPU and 16GB of RAM) and the display is solid, but as someone who uses desktops primarily, I wasn’t used to using such a large laptop (although it was quite thin).

Dell M3800 - Mint Login Screen

Mint worked pretty well, but there was a frustrating issue with the clickpad. Sometimes I was unable to select a piece of text on the first try. On a second (or sometimes third attempt) it worked fine, but I could never get the behavior to go away entirely. I have found hints on the Intertoobz that suggest it is a known issue with Cinnamon, so perhaps it will be addressed in 17.2.

My main issue with the unit, outside of the size, was the battery life. I could sit and watch the battery percentage drop, about one percent a minute. This was in light duty mode, such as writing e-mails and browsing the web. While 100 minutes of battery life isn’t terrible, it is less than half of what I am used to.

I took a guess that part of the problem could be in the weird hybrid video controller setup they use. There is both an NVidia card and an Intel card in the unit. I installed bumblebee and that seemed to help some, but it didn’t make the power issue go away.

[Note: as an aside, many thanks to Arch for having such amazing documentation]

Dell M3800 - Mint desktop

Overall, if I was looking for a laptop to replace my desktops, I would have tried to stick with it longer. But the size coupled with the battery issues made me send it back. I was still in love with the XPS 13 so I decided to just wait until they supported Linux on it.


I really like visiting the state of Oregon, from Portland all the way down to Roseburg. While I haven’t been to the eastern side, the western side is beautiful, and there is also a lot of OpenNMS history there. The City of Portland uses OpenNMS, as does Earthlink Business Solutions (currently across the river in Vancouver, WA, but with a data center in Portland). Ken Eshelby, who works for the State of Oregon, has one of the most amazing installations of OpenNMS I’ve ever seen (he takes my definition of OpenNMS as a platform very seriously).

But my heart always resides with Oregon State University down in Corvallis. When I first started out on my own back in 2002, OSU was the second organization to purchase a Greenlight project. They’ve been using it for ten years, and last week they invited me back to do a “Tune My OpenNMS” project in order to get them on the latest version and to show them ways to maximize how they are using the tool.

The “Tune My OpenNMS” project (once called “Pimp My OpenNMS” but we had a client warn that he could never submit a purchase request with the word “pimp” in it) is three days long, so I decided to come in a day early to visit friends down in the Peoples Republic of Eugene. Sunday we sat outside drinking microbrews and feasting on salmon caught off the coast, and on Monday I got to go hiking in the Tamolitch Valley.

The area has both old and new growth forests, and there are some pretty extensive lava flows. Here’s a picture of my friend Kate where you can see the lava starting to be overgrown with brush.

The path we took followed the McKenzie River, which made me wish I had brought along my waders and fly rod, but since I didn’t I had to content myself with just pointing out those places where the monster trout would live.

Our destination was a place called The Blue Pool. This is a small pool that contains (so I learned) a lot of dissolved silica. While it shows up as white on the tree trunks and rocks downstream, in the still water of the pool it shows up as an incredibly rich blue (as it scatters blue light). There were once falls on one side of the pool but a lava flow covered them up. There is still water flowing under the rock and it comes out under the water, so they call it a “dry” falls.

I said goodbye and headed up to Corvallis Monday night. On Tuesday I met with Joel Burks at OSU and we got down to business migrating their existing 1.6.7 install to 1.10.5. For lunch we met Bill Ayres (OGP) who was one of the leaders of the OSU OpenNMS project until he retired, and the three of us had a lot of fun.

One night we drove out to Newport to eat seafood at Local Ocean. It was amazing.

I also got to see the controversial mural located downtown that illustrates atrocities committed by Chinese soldiers against Tibetan monks.

I thought it was quite the coincidence that I was in Corvallis when a news story broke that wasn’t about OSU athletic scores.

I didn’t get a paper accepted at OSCON this year so this was my only trip to the area, and I had missed it. I hope to return soon.

Super Bowl XLVI

Congratulations to the world champion New York Giants for winning Super Bowl XLVI.

I’ve been an NFL fan since I was nine years old. I was born the year before the first Super Bowl, and my team, the Pittsburgh Steelers, won Super Bowl IX on my ninth birthday, so I’ve always had a soft spot for the game (NFL expansion has made sure that my birthday will never fall on Super Bowl Sunday again). In all those years of watching football, I have never seen the situation where one team wanted to let the other score a touchdown, and to watch an offensive player tried his damnedest not to score one.

Of course, it wouldn’t be the Super Bowl without Papa Johns Pizza, and this year I got to order from the brand new Pittsboro store. Here is a screenshot of the OpenNMS store monitoring instance:

I ordered our pies about 2pm for a 5pm pickup and it went flawlessly (included getting 50% off by using the promotion code “CANIAC” since the Carolina Hurricanes had won their last game). We had some of the gang from the office and other friends over, and it was a nice respite from trying to get OpenNMS 1.10 out the door.

The commercials could have been better, however.

Sys-con Media Rates OpenNMS Number One

I’ve spent the first half of this week teaching an OpenNMS course. It’s a lot of fun and we have a great class (the next one is in May if you are interested).

While I was talking to the students, one pointed out that he was able to get permission to attend the class because he was going to learn about “the number one systems management tool” available. While I didn’t disagree with him (grin) I did ask how he came by that information, and he pointed me to a Sys-con Media article published last month that compared 11 systems management offerings, both open and closed source.

I had missed this article, but it ends with:

OpenNMS 1.6.10 scores better than the competition, and is thus a better server monitoring software. Its basic features include a faster configuration process, web interface, compatibility and advanced features, such as “better automatic corrective actions”. OpenNMS is also free of cost.

OpenNMS rated a 7 out of 9. Of the other 10 products reviewed, there was one 3, three 4’s, four 5’s and two 6’s.

I, of course, think OpenNMS should have scored a 9 out of 9. The two areas that we missed were “Mobile Access” and “Integrated Maps”. Since the review was against 1.6.10 (versus 1.8.10 which is the latest stable version available) it didn’t have access to the iPhone app, and we have both greatly improved the built-in OpenNMS mapping feature as well as added an integration with Google Maps, Mapquest and Open Street Map for the remote pollers.

It was still nice to come out on top, however, and I hope this gets even more people interested in OpenNMS.

Using OpenNMS Remote Monitor for Remote Polling

Kevin Sonney has posted a detailed blog entry on how he is using the OpenNMS remote monitor as a remote poller.

Now one might ask: what is the difference between remote monitoring and remote polling?

Well, when the feature was written, it was created for The Permanente Management Group in northern California. They had centralized a number of services at a data center in Oakland, and doctors and other staff need to access those services from any one of 350 or so offices and clinics around the state. They needed a way to monitor them from the point of view of the remote site: hence remote monitoring. Papa Johns uses this as well: monitor a small number of centralized services from a large number of remote sites (2743 at last count).

However, what a lot of people think of when they hear “remote monitoring” is actually remote polling: how can I monitor localized services at a remote location? Kevin’s post details that.

In OpenNMS 2.0 the plan is to have multi-level distributed monitoring and data collection, but until that is complete Kevin’s instructions demonstrate how to do it with the current code. Many thanks to him for taking the time to write it up.