The Brightspeed Saga

While I love living “out in the country” I often envy my urban friends for their network connectivity. When I moved out to the farm in 1999 the only “high speed” access was satellite, and even that required a modem and a phone line. I was overjoyed when Embarq finally deployed DSL to my house, and while 5bps down might not seem like much these days it seemed heaven-sent back then.

Jump forward 20 years and Embarq became Centurylink which is now Brightspeed. I had a pretty poor opinion of Centurylink (or as I called them CenturyStink) but high hopes for Brightspeed when they bought Centurylink’s ILEC business in our area, but they have been disappointing. Here is that story.

Both my wife and I work from home, and when our DSL circuit is working it works well enough for us to get our jobs done. At 11Mps down and 640kbps up it doesn’t even qualify as “broadband” but it is a trade off I’m willing to make in order to live where I live.

Starting back in early November we began to have issues with the quality of the DSL connection. Quality issues are always frustrating since the support technicians at the provider never seem to have the tools to properly measure it. Instead they just tell me the circuit is “up” so I should be satisfied, even though I tell them that while it is up, it is unusable.

The issue was high latency and packet loss. Latency is a measure of the time it takes information to travel through the network and packet loss indicates that some of that information never makes it to its destination. The protocols used in networking will automatically deal with packet loss by sending the information again, but the more this happens the worse the experience is for the user. Things that can handle packet loss gracefully like e-mail, web pages and chat just seem very slow, while anything that requires a more steady flow of information like video or gaming just don’t work at all.

Having done network monitoring for much of my professional life, I monitor the quality of my DSL circuit by attempting to reach the 8.8.8.8 IP address, which is a highly available DNS server run by Google. Here is a recent graph:

Now normally the graph should be green and pretty much focused around 45ms. This one was all over the place. I asked my neighbor to execute a ping to the same IP address and her connection was working fine, so I assumed it was an issue specific to me.

Trying to get support from Brightspeed was very frustrating. As I mentioned above they just tend to tell me everything is okay. I even reached out to one of my LinkedIn contacts who is an executive vice president at Brightspeed for help, and I think he was responsible for a ticket being opened on November 7th in the “BRSPD Customer Advocacy (Execs)” queue. I really appreciated the effort but it didn’t help with getting my issue resolved.

Just after Thanksgiving I called again and I was told that the problem was that my modem was wrong and doesn’t work with the DSL circuit we have, even though it was the model they sent to me and it had been working fine up until November. In any case they said they would send me a replacement and it would arrive in two days.

Ten days later I call back and ask, hey, where is my modem? They told me it was still “in process” but since I’ve been waiting so long they’ll overnight one to me. It shows up the next day and of course doesn’t fix the issue, but it has newer software than my other modem and reports the status of the DSL circuit. On the web page of the device this is usually represented by the word POOR in dark red, but sometimes it would improve to MARGINAL in a slightly lighter red. I call and explain this to Brightspeed, and after dealing with this for over two months they agree to send out a technician in two days.

When I finally get the e-mail confirming the appointment, it is for the following Thursday, January 5th, eight days away. That also happens to be when we are closing on a new house, so I can’t be here to meet with them. For some appointments they show up early, so I didn’t change it right away, but when they hadn’t shown up by Tuesday I decided to reschedule it.

I went to the e-mail and clicked on the link to reschedule and got sent to the Centurylink site. Of course they wanted me to confirm my account, but nothing I typed in: phone number, e-mail, or account number, worked because I no longer have a Centurylink account. (sigh)

[Note: it looks like this has been corrected, finally, on the Brightspeed website. Not sure about links in e-mails]

In the process I did find out that my Brightspeed account number ends in “666” so perhaps that is indicative of something.

I eventually ended up calling support once again. I believe it would take the average caller about six minutes to reach a human through their system, as it prompts you for a variety of things before allowing you to speak to a person, but I had been calling for over two months so I can speedrun the thing in about four and half minutes by pressing buttons before the prompt is finished.

The person I talked to about rescheduling the appointment kept me on hold for about 30 minutes before telling me that the whole dispatch system for technicians was down and that she would call me back in two hours. She never did.

The next day I made one more attempt to reschedule the appointment, but was told that the next available appointment was so far out in the future that I should just keep it, since the technician won’t need to enter the house. I left a long letter taped next to the demarcation box on my house with a detailed description of the problem, and hoped for the best.

Unfortunately, they sent out Brandon. To my knowledge there are only two technicians assigned to our rather large county: Brandon and Elton. I much prefer working with Elton since Brandon doesn’t really seem to be the kind of person who does a deep dive into the problem, but I recently learned that Elton has moved into the back office and wasn’t doing service calls anymore.

As I feared, Brandon marked the issue as closed without fixing it. Once again into the support phone queue, where I was told that he had run a test “for five minutes” and my circuit was fine. (sigh)

I did get a text asking if my problem was resolved, to which I said “NO!” and I was later contacted by a person from Brightspeed to follow up. After a very long conversation she offered to send someone else out, and that person arrived yesterday.

Philip, who is based out of Wake County (one county to the east of us) showed up promptly at 8am and within ten minutes had diagnosed a grounding issue with the wires coming to our house. In about 45 minutes he had repaired it, but he warned me that there was also an outage in the area which would explain my now 900ms ping times (but no packet loss). I trusted him that it would eventually resolve and about 30 minutes later things were much, much better.

You can see where the network was bad before Philip showed up, the gap where he was working on the system, and then the return to a more expected quality of service.

It still isn’t perfect. I’m seeing a lot of jitter from time to time which is indicated by the spikes, but for the most part the user experience is fine. I was able to participate in our departmental weekly video call without issue yesterday for the first time in months.

And that’s was really bothers me the most. For nearly three months Brightspeed was gaslighting me that my service was fine, when, as most IT professionals would expect, it turned out to be a physical layer problem. In retrospect it makes sense since we’ve been having an especially wet winter and that would have caused the grounding issue to be amplified.

I figure I spent between 40 and 60 hours actively involved in getting this addressed, and that is time I’ll never get back.

Of course it could be worse. The local newspaper published a story about a community in Chatham County that was without service from Brightspeed for a total of 51 days. At least our connection was usable enough that it only required a few trips to the public library for access during important deadlines.

There is some good news in that same newspaper issue that some attempts are being made to help those of us in rural areas get broadband. Some of you may be thinking Starlink, but I was on their waiting list for two and a half years without getting my equipment and when they pushed it out to late 2023 I just gave up and asked for my deposit back.

I am not a huge “we need to regulate everything” kind of guy, but broadband has become one of those services that is so important and that the free market has failed to provide that I would welcome government involvement in getting this issue addressed. But so far the communications lobby has been strong enough to prevent any kind of oversight, so I won’t hold my breath.

Update on Expensify

I recently posted a rant on how a vendor we use, Expensify, appeared to be exposing confidential data to workers with the Amazon Mechanical Turk service. In response to the general outcry, they posted a detailed explanation on their blog.

It did little to change my mind.

So apparently what happened is that they used to use the Mechanical Turk from 2009 to 2012, so if you we a customer back then your information was disclosed to those third party workers. Then they stopped, supposedly using some other, similar, in-house system.

But, some genius there decided that the best way for certain customers to insure their receipts were truly private was to have them use the Mechanical Turk with their own staff. I covered that in my first post and it is so complex it hardly registers as a solution.

Of course, they decided to test this new “solution” starting the day before the American Thanksgiving holiday. This was done using receipts from “non-paying customers”. While we pay to use the service (not for much longer), if you were trying it out for free your receipts were exposed to Mechanical Turk workers. Heh, if you aren’t paying for the product you are the product. The post goes on to talk about the security of the Mechanical Turk service, which was surprising because they went on and on about how they didn’t use it.

What really angered me was this paragraph:

The company was away with our families and trying hard to be responsive, while also making the most of a rare opportunity to be with our loved ones. Accordingly, this vacuum of information provided by the company was filled with a variety of well-intentioned but inaccurate theories that generated a bunch of compounding, exaggerated fears. As a family-friendly business we try hard to separate work life from home life, and in this case that separation came at a substantial cost.

Well, boo hoo. If you truly cared about your employees you wouldn’t start a major beta test the day before a big holiday. I spent my holiday worrying about my employees’ personal data possibly being exposed through the Expensify service. Thanks for that.

What pisses me off the most is this condescending Silicon Valley speak that their lack of transparency is somehow our fault. That our fears are just “exaggerated”. When Ryan Schaffer posted on Quora that nothing personal is included on receipts, he demonstrated a tremendous lack of understanding about something on which he should be an expert. As they turn this new leaf and try to be more transparent, I noticed he deleted his answer from the Quora question.

Smells like a cover up to me.

Look, I know that being from North Carolina I can’t possibly understand all the nuances of the brain-heavy Valley, but if Expensify truly does have a “patented, award-winning” methodology for scanning receipts, why don’t they just make that available to their customers instead of using the Turk? This long-winded defense of the Turk seems like they are protesting too much. Something doesn’t make sense here.

I’ve told my folks to stop using SmartScan and that we would move away from Expensify at the end of the year. If you use, or are planning to use, Expensify you should deeply consider whether or not this is a company you want to associate with and if they will act in your best interests.

I decided the answer was “no”.

Why the FCC’s Title II is so Important (Spectrum Rant)

Here is a rant about Time Warner/Charter/Spectrum or whatever the heck they call themselves these days. It illustrates how this large company can have a huge negative impact on a small business, and why treating Internet providers as common carriers is so important.

Our company wouldn’t exist without the Internet. Outside of the fact that our products are mainly used to monitor Internet resources, we host a number of servers from our office and about half of the staff works remotely so we rely on the Internet to communicate and coordinate.

Back in 2012 I contracted with Time Warner to provide Internet access to our office. We had fiber to the building and while our service was considerably more expensive than coax, I liked the fact that it was symmetrical and expandable. We started of with 20 Mbps but soon increased that to 50 Mbps. Over five years we only had one outage, due to a misconfiguration of our Customer Premise Equipment (CPE), and they corrected it within 20 minutes. I love the fact that when you called in the person who answered the phone understood terms like “duplex” and they were always very helpful.

Note the scenario: happy customer who is happy paying a premium for enterprise-level service.

Now let me tell you why all that goodwill has gone away.

Earlier this year we decided to move our office from Pittsboro, NC to Apex, NC. The first thing I did was contact Time Warner (well, Charter at the time) to insure that they could provide fiber to the new location. They said they could, although it would take 45 to 60 days. As our new office space needed to be completed, we were targeting an April 1st move in date anyway, so on February 15th I placed the order for the new service. At best, it would be available on the 1st and at worst it would be ready by the 15th. We told the old landlord we’d be out by April 30th just in case and to give us more time to move.

Finally, Spectrum doubled our speed and cut the price in half. I was feeling pretty good about the whole thing.

The feeling didn’t last.

As we got closer to April, things started to go wrong, most of it due to the fact that Spectrum is now such a behemoth that they have no idea what they are doing. In order to get fiber into our new building, they needed what is called a “Right of Entry”. They sent it to our landlord who promptly completed the form and sent it back. However, that person didn’t let the project manager know the form had been received, so he did absolutely nothing. Ten days (!) later I get a note that our build out had been suspended because of the lack of the ROE form. A form, I should point out, that was sent to them, twice.

At the end of March I’m told that our new date is May 11th. I’m unhappy – due to their poor processes I now have a new office that I can’t use for six weeks (remember, we took possession and started paying rent on April 1st). We also had to be out of the old office by the end of April. Luckily I work with a great team that is able to be productive when working from home, so I decided to suck it up and live with it.

On April 12th I get an update – the new date for the end of construction is now May 15th due to processes within Spectrum taking too long to finalize the work with a contractor. Now the actual date we’ll have Internet has been pushed out to the week of May 29th.

I am livid. By this point I’m ready to switch to the other option, AT&T. Unfortunately, they also need 45 to 60 days for service installation so I realize at this point I’m stuck with Spectrum.

I ask my salesperson for options and he suggests we get coax installed for a month (for a fee, of course). Since our office is right next to a large housing development they can get coax in the following week. I sign off on it.

It didn’t happen. When May arrived some of us started working in the new office mooching off the neighbor’s Wi-Fi from AT&T (with permission of course). I ended up traveling for a couple of weeks so I completely forgot about the coax option (it’s not like Spectrum was keeping me updated on anything – I’d have to reach out to them for an update). I did get a note on May 10th that all construction had been completed for the fiber and another note on May 18th that our new install date was June 2nd.

(sigh)

So, 45 days late, we have a firm install date. Wonderful.

Imagine how I felt when on the 24th of May I received a note that more construction was needed and that it would be pushed out another 30 days at least. When I get extremely angry I refer to it as going “non-linear” as that how fast my blood pressure rises. As I was ranting to pretty much everyone I’d ever interacted with at Spectrum it dawned on me that this could be for the coax order. Turns out that was the case. Apparently our crack project manager on the coax side decided to route our service from a point several miles away instead of from the one nearly across the street. This is why it was delayed and why the construction was needed. By this time we are about a week out from having fiber so I canceled the order. I did get a very apologetic call from the coax salesperson which I appreciated (under Spectrum, fiber [Enterprise] is handled by one sales team and coax [Business] is handled by another), and I made it clear that I’d be okay with everything as long as the fiber was delivered as promised on the 2nd.

It was. Around noon on June 2nd we had our 100 Mbps service and on the 3rd we moved all of our devices from the old office in Pittsboro to the new one in Apex. I informed my salesperson that they could disconnect the old service and despite all of the problems, I was happy with the new service.

So the whole process cost me two months rent and a few years off my life, but it was finally over.

Not so fast – the other shoe fell today.

I get an e-mail that I need to confirm my disconnect request. That didn’t bother me, in fact I appreciated it, but what did bother me was an additional note that it would be done within 30 days. When I replied I asked for clarification – would I be *paying* for the service I wasn’t using until they could disconnect it? The answer was “yes”.

I experienced a new word – apoplectic.

Due to the fact that the bureaucracy behind the new merged Spectrum company is so bad, I’m out nearly ten thousand dollars. That is the real money – it’s probably cost us twice that again in lost productivity from lack of network access and dealing with them throughout this process. We’re not one of those companies that is too big to fail so this really impacts us negatively. Had it been explained to me that I’d have to pay for the service until it was disconnected, I would have put the disconnect order in a month ago, but then had I used the date I was originally promised, our servers would have been off-line for over a month. That would have been catastrophic to our company.

Finally, I’ve gone from a happy customer to an extremely pissed off one who will be actively looking for options. Based on my experience I would suggest any business looking for network access look elsewhere.

Access to the Internet has become as important as other utilities such as electricity, water and sewer and just like those utilities it needs to be regulated as one. This is why the decision by the new industry-picked head of the FCC to reverse the decision to classify Internet access under Title II as a “common carrier” is so devastating to businesses like mine. Our company is small, yet we put millions of dollars into the local economy each year. You multiply that by the number of other small businesses and it can have a great impact to any community. Barriers put up by companies like Spectrum demonstrate that they can’t self-regulate and the government needs to take a firmer hand (and this is coming from a left-leaning libertarian).

I will be protesting that final bill for Internet access and I would welcome any advice on how to deal with a company like Spectrum. Let’s hope that there is a change soon so that other businesses can focus on creating value and not have to deal with the crap we had to endure.

I’m not holding my breath.

Monitoring? Meh.

Recently, I was talking to a person in the tech industry and describing all of the cool things we are doing with OpenNMS, when he kind of cut me off and went “Oh, monitoring? Meh.”

Well, I can’t remember if there was an actual “meh” but that’s how it came across, and I’m afraid the reaction is probably more common that I would think. Monitoring isn’t sexy, but it surprises me that people can’t see how critical it will be to the future of any business.

IoT Devices Over Time

While forecasts vary, by 2020 there are expected to be over 30 billion devices on the Internet, and that figure will skyrocket to over 75 billion by 2025. Just knowing what is connected to your business network is going to become critical, as well as making sure it belongs there in the first place and, if so, is functioning properly.

Outside of the obvious security concerns, as people began to transact business more and more through devices rather than people, faults in those devices will directly impact revenue as people search for other options when faced with a bad experience.

Here are a couple of examples.

One of the greatest inventions in my lifetime is the ability to buy fuel at the pump. You just pull up, swipe your card, pump and then leave. You used to have to pay inside, and some places made you pay first which meant two trips in if you were paying by credit card. It could be cold or rainy, and not only did you have to wait in line behind people buying food or lottery tickets, you had to leave your car out by the pump possibly blocking the next customer.

The only problem I’ve experienced with this process concerns the receipt. Quite frequently I need a receipt, but it seems the pumps I choose are always out of paper. The little red indicator mark when the paper roll is almost finished isn’t visible to the cashier since there really isn’t one out by the pump. It is frustrating, but it is not like I have a choice at the moment. If there was some way to monitor the pump for a “low paper” alarm, it would improve my shopping experience.

One shopping experience that did result in my leaving the store without a purchase happened yesterday at a Lowe’s Home Improvement store. I needed some florescent lights for the new office so I went by on my way home. I picked up four bulbs (two that I needed and two spares) and went to the checkout area.

I walked past several unmanned cash registers until I got to the “Self Checkout” section, which was the only thing open. Of the four machines, two had red blinking lights on them (that are green when things are functioning normally) and the one lone, overworked cashier was doing her best to help people out. I usually don’t mind using Self Checkout and when I noticed one of the two machines was open (everyone else was waiting for the attention of the lone cashier) I went to it and started my purchase.

I scanned my “My Lowe’s” card and then the first bulb. “Eight ninety-five” piped up the voice and I placed it in a bag.

Here is where the problems started. First, I hate the fact that with these Self Checkout kiosks they don’t trust you to use a “quantity” key. I was buying four identical items but I was required to scan each one. Next, the bulb was light enough that it didn’t register as having been bagged, so the interface yelled at me and presented me with a button marked “Skip Bagging Item?”.

I sighed and, having no other option, hit the button. I then went on to scan the next three bulbs. However, as I bagged the fourth bulb, the scale must have started working since the whole unit went into some kind of alarm mode, screeching “Unidentified Object in the Bagging Area!” and the screen was locked until the cashier had time to come and fix it.

I looked around the area, and by this time all four kiosks had a flashing red light, there were at least three shoppers lined up to use them in addition to those of us already there, and our valiant cashier was busy helping a guy ring up his plumbing supply purchase which consisted of a ton of small copper fittings which most likely wouldn’t be registered by the scale.

I gave up. I picked up my bulbs and returned them to the Lighting section, passing three employees in the customer service area helping zero customers. Before I reached the car I’d ordered the same bulbs on Amazon at a fraction of the price, and they’ll be here on Friday.

Yes, I’m complaining, but how could monitoring have helped here? First, there is some sort of monitoring – those little red lights. When they all light up you would assume someone, or perhaps multiple someones, would come by to help. A monitoring system could have made sure that happened by using an additional notification system outside of the lights, and escalating it until the problem was addressed.

A more long term solution would be to collect information on the purchasing experience and the problems people encountered and to make changes to the automated kiosk software. I’m certain that Lowe’s didn’t write that software but instead bought it, and like most proprietary software solutions they now have to fit their processes to the application instead of the other way around. It probably wasn’t designed for a store that sells a lot of small, light things which is central to the issues I have using it.

With the rise of IoT devices, robotics and other forms of automation, monitoring is going to become extremely important. Lowe’s lost out on a $40 sale, but think of something like an assembly line where a problem could result in the loss of thousands of dollars a minute. Our goal at OpenNMS is to be ready for it, and to build products that make people go “Monitoring? Oh yeah!”.

Open Core Returns from the Dead (sigh)

The last 18 months of my life have been delightfully free of “open core” companies. These were companies who pretended to be “open source”, at least in their marketing materials, yet their business model was based on selling “enterprise extensions” which consisted of proprietary software that actually had the features you wanted. Basically, the open source piece was a loss leader to get you to buy the commercial edition, and as Brian Prentice pointed out so eloquently there was no real difference between “open core” and traditional closed source software. We like to call these businesses “fauxpen source“.

Customers realized this as well, which lead most open core companies to switch their tactics. While many still maintain an open source project, they have removed the term “open source” from their websites and most of their marketing (often replacing it with “open architecture”). I’m happy with this, as it allows true open source companies like OpenNMS and Nextcloud to differentiate ourselves while allowing these other companies to still produce open source software without misleading the market.

But lately I’ve been introduced to two new licenses that offer access to the source code without meeting the ten requirements of the Open Source Definition. These licenses further muddy the waters due to giving access to the code without including the freedoms of truly open software.

The first case was from Monty Widenius, who announced a proprietary Business Source License (BSL) for some of the MariaDB products. Monty was the guy who earned €16.6 million by selling MySQL to Sun and then got upset when Sun got bought by Oracle. Apparently, he seems to be unhappy that he isn’t earning enough money from his fork of MySQL products so he wants to create commercial software but not call it that.

The BSL, or as I call it, the “Rape of Large Companies License” allows the developer to offer the code up for use for free unless you cross some sort of arbitrary threshold, also set by the developer. In three years that code will revert to an OSI approved license, in this case the GPLv2, and if you are above the usage threshold then you don’t have to pay anymore.

I’m not sure what his goals are here, outside of running a commercial software business while paying lip service to open source software. Perhaps he hopes to get people to contribute to BSL licensed projects as long as their use case is small enough not to cross the “pay me” threshold, but more likely he just wants to ride on the coattails of the success of open source software without committing to it.

I learned of another such license called the Fair Source License (FSL) from a post by Ben Boyter who writes the Searchcode Server. Ben, at least, is a lot more up front about his reasons for adding a “GPL Timebomb” to his code. Initially, the code is published under the FSL but with a switch to the GPLv3 in three years. He isn’t expecting contributions and instead has offered up the code simply so it can be audited for back doors. As this is one of the more powerful features of open source software I applaud him for doing it, but I really wish he hadn’t used the term “bomb”. I have to deal with terms like “GPL poisoning” enough in my business that negative words like that just tend to scare people. He should have called it “Happy Fun Lucky GPL Gift Giving Time!”

Look, I’m all for anything that gets more code out there under an OSI-approved license but, c’mon, three years is a lifetime in this industry. Enterprise customers, who would be most affected by this license, will still have to approach the buying decision as if a BSL or FSL licensed application were commercial software. Even with the three year revenue window, it is unclear what happens if, say, a huge security bug is discovered three years out. Does the code to fix that bug restart the clock?

The whole process is confusing and doesn’t help the cause of open source software. I think open source is awesome and extremely powerful, and when I see things like this I’m almost insulted, as if the developer is saying “when I’m done you are welcome to my leftovers”. Instead of announcing a future switch to an open source license years in advance, they should just open it when they are ready, like id Software does with the Doom engine.

I’m giving a talk at All Things Open about running a truly open source business, the core point of which is that you can’t have an open source business with a business model based around selling software. No matter how you dress it up by either calling it “open core” or “business source” it is still proprietary software.

Nagios XI vs. OpenNMS Meridian – the Return of the FUD

It seems like our friends over at Nagios have been watching a little too much election coverage this year, and they’ve updated their “Nagios vs. OpenNMS” document with even more rhetoric and misinformation.

As my three readers may recall, back in 2011 I tore apart the first version of this document. Now they have decided to update it to target our Meridian™ version.

Let’s see how they did (please look at it and follow along as it is quite amusing).

The first misleading bit is the opening paragraph with the phrase “most widely used open-source monitoring project in the world”. Now, granted, they do indicate that means “Nagios Core” but it seems a little disingenuous since what they are selling is Nagios XI, which is much different.

Nagios XI is not open source. It is published under the “Nagios Open Software License” which is about as proprietary as they get. I’m not even sure why the word “open” was added, except to further mislead people into thinking it is open source. The license contains clauses like “The Software may not be Forked” and “The Software may only be used in conjunction with products, projects, and other software distributed by the Company.” Think about it, you can’t even integrate Nagios XI with, say, a home grown trouble ticketing system without violating the license. Doesn’t sound very “open” at all. OpenNMS Meridian is published under the AGPLv3, or a similar proprietary license should your organization have an issue with the AGPL. You don’t have that choice with Nagios XI.

Next, let’s check out the price. The OpenNMS Group has always published its prices on-line. One instance of Meridian, which includes support in the form of access to our “Connect” community, is $6,000. They have it listed as $25,995, which is the price should you choose the much more intensive “Prime” support option. I’m not sure why they didn’t just choose our most expensive product, Ultra Support with the 24×7 option, to make them seem even better.

Nagios XI Node Limitation

Also, note the fine print “Price based on one instance of XI with 220 nodes/devices”. There is no device limit with OpenNMS Meridian. So let’s be clear, for $6000 you get access to the Meridian software under an open source license versus $5000 to monitor 220 nodes with extreme limitations on your rights.

Our smaller customers tend to have around 2000 devices, which means to manage that with Nagios XI you would need roughly ten instances costing nearly $50,000 (using the math presented in this document). And from the experience we’ve heard with customers coming to us from Nagios, the reason it is limited to so few nodes is that you probably can’t run much more on a single instance of Nagios XI. Compare that to OpenNMS where we have customers with over 100,000 devices in a single instance (and they’ve been running it for years).

We also price OpenNMS as a platform. You get everything: trouble-ticketing integration, graphing, reporting, etc. in one application. It looks like Nagios has decided to nickel and dime you for logs, etc. and a thing called “Nagios Fusion” which you’ll need to manage your growing number of Nagios instances since it won’t natively scale. And remember, due to the license you are forbidden from using the software with your own tools.

I especially had to laugh at the “You Speak, We Listen” part. If you have a feature or change you need, if you ask nicely they might make it for you. With OpenNMS Meridian you are free to make any changes you need since it is 100% open source, and with our open issue tracker we address dozens of user requests each point release.

Finally, there is the feature comparison, which at a minimum is misleading and is often just blatantly false. Almost every feature marked as lacking in Meridian exists, and at a level far beyond what Nagios XI can provide. Seriously, is it really objective to state that OpenNMS doesn’t support Nagvis, a specific tool that even has “Nagios” in the name?

Nagvis

I had to laugh at the hubris. They obviously didn’t Google “opennms nagvis“, because, guess what? There has been an OpenNMS Nagvis integration for some time now, contributed by our community. Just in case you were wondering, we have an integration with Network Weathermap as well.

Nagios is just another proprietary software product that wants to lock you into its ecosystem, and this is just a shameful attempt to monetize an application that is long past its prime. Heck, it was the inability of the Nagios leadership to get along with others that resulted in the very popular Icinga fork, and with it Nagios lost a lot of contribution that helped make up its “Thousands of Free Add-Ons” (and the way Nagios took over the community lead plug-in site was also poorly handled). Plus, many of those add-ons won’t scale in an enterprise environment, which probably lead to the 220 device limit.

Compare that to OpenNMS. We not only want to encourage you integrate with other products, we do a lot of it for you. OpenNMS has great graphing, but we also created the first third party plug-in for Grafana. When it comes to mapping, OpenNMS is on the leading edge, with a focus on various topology views that can ultimately handle millions of devices in a fashion that is actually usable. Need to see a Layer 2 topology? Choose the “enhanced linkd view”. Run VMware and Vcenter? It is simple to import all of your machines and see them in a view that shows hosts, guests and network storage. Plus the unique ability to focus on just those devices of interest allows you to use a map with hundreds of thousands if not millions of nodes.

Nagios Map

Compare that to the Nagios map screenshot where it looks like “localhost” is having some issues. Oh no, not localhost! That’s like, all of my machines.

As for “Business Process Intelligence” I’ve been told that the Nagios XI version is like our Business Service Monitor “Except BSM is more featureful, and has a significantly better UI/UX”. Need real Business Intelligence? OpenNMS has Red Hat Drools support, the open source leader, built right into the product.

We also support integration with popular Trouble Ticketing systems such as Request Tracker, Jira, OTRS and Remedy. And the kicker is that you can also run any Nagios check script natively in OpenNMS using the “System Execute Monitor“, but once you get used to the OpenNMS platform, why would you?

I’m not really sure why Nagios goes out of its way to spread fear, uncertainty and doubt about OpenNMS. We rarely compete in the same markets. I’m sure that Sunrise Community Banks get their money’s worth from Nagios, and for companies like NRS Small Business Solutions, Nagios might be a good fit. But if you have enterprise and carrier-level requirements, there is no way Nagios will work for you in the long term.

When a company does something like this to mislead, from wrong information about our product to using terms like “open” when they mean “closed”, it shows you what they think of their competition. What does it say about what they think about their customers?

Framing the Net Neutrality Debate

Sorry for the delay in commenting on this, but I’ve been sick with the flu and I’m just now getting back to normal.

Back on February 26th, the FCC voted to treat the Internet as a utility. As a huge fan of “Net Neutrality” I was cautiously optimistic about this, but I was saddened by the fact that the 3-2 vote was along party lines. With the current dysfunctional US government, I was hoping that something as important as a free and open Internet would not be politicized.

Those who would try to control the Internet for their own agenda were quick to respond. Verizon issued a reply that looked like it was created on an old typewriter, implying that ideas created in 1934 (the law that formed the basis of the FCC decision) couldn’t be useful in a modern world. It’s an informal fallacy – instead of trying to describe why the world would be better off with Verizon in control of the Internet, a much harder proposition, they opted to throw a bunch of rhetoric at the problem.

Those against Net Neutrality are going to try to frame the issue as “anti-capitalist”. The problem is that a key component of free markets is “easy entry and exit”. The idea is that if one company is making a profit, competitors will enter in to the marketplace until the overall profit reaches zero. The problem with utilities such as the Internet, electricity and water is that it is not easy to enter or exit the market, which creates barriers to a truly free marketplace. While I can argue that it is uniquely American for a large corporation to try and protect its unfair advantages, it is also anti-capitalist and government should exist to maintain a level playing field.

No, what Verizon dreams about is becoming the Enron of the Internet. I managed to get my hands on a leaked, new “Venron” logo:

Enron demonstrated the problems when you try to mix in greed with a utility. Due to companies such as Enron exerting undue influence in politics, the decision was made to deregulate the generation of electricity in California. Everyone used the same rhetoric being used in the Net Neutrality debate: free markets are the best for the people and “trust us” – why would we want to hurt our own customers? This resulted in skyrocketing energy prices and rolling blackouts throughout the State.

Internet companies such as Verizon are using those same tactics to build opposition to Net Neutrality. Their pet politicians, such as Senator Ted Cruz of Texas (the State responsible for Enron) have been hard at work trying to tie a free Internet to the idea of “too much government” and you can expect them to compare it to everything from the Affordable Care Act to Climate Change.

We need to respond with the simple mantra that a vote against Net Neutrality is a vote for an “Enron of the Internet”. It’s as easy and straightforward as that. Net Neutrality means that no one company, or cabal of companies, can control the Internet – not even the government. It is a vote for freedom and democracy, and anyone who is against that is against the ideals that created our country in the first place.

Yeah, I know that the Constitution is a really old document (even older than 1934) but it has held up pretty well. Let’s make sure that the opponents of a free Internet aren’t allow to disgrace it as well.

Dell, Rhymes with Fail

Yes, I am a bit frustrated at the moment. This post is something of a plea that someone within the huge organization known as the Dell Computer company has a clue and can help me out. Before you think I’m just a big hater, here is a shot of one of our computer racks:

As you can tell, we do use a lot of Dell hardware (and yes, there is an HP box squeezed in the middle there).

I work hard. As a result of that, I feel I deserve nice things, and what I really want right now is a nice laptop. But I want a laptop that runs Linux well.

I’ve looked at the systems from System76, but I want a higher density screen than they offer. I would look at the new X1 Carbon from Lenovo, but I’m still angry at them for Stoopidfish, and while I plan to wipe any laptop I get from any vendor I still think it will be some time before I can give them money.

No, I like to support Linux-friendly vendors, so I recently ordered the Dell M3800 laptop, Ubuntu edition. I ordered it on February 2nd.

A couple of my three blog readers have contacted me eager for my review, but I wasn’t able to publish it because I have yet to receive the laptop. In fact, it appears my original order has been canceled. Here is the story.

I placed the order on the 2nd, and got an estimated delivery date of the 18th. That would have been perfect as I would have the new machine just before SCaLE. Unfortunately, it was not to be, and my estimated delivery date was pushed out until the 26th.

Well, the 26th came and went with no update from Dell. I finally decided to contact them and I was told they would expedite my case.

This week I was told that “I regret to inform you that we are not able to process the order 769577335 due to configuration mismatch.”

WTF?

You drag me along for a month and then tell me that there is a “configuration mismatch”? Now I have to reorder and go through the whole process again? Plus, there was no “mea culpa” and no offer of, heck, free shipping or an expedited order – just “so sorry, try again”.

Grrrr.

Like an idiot, I decided to try again.

I got to the order page, and that’s when I found out what the magical “configuration mismatch” was. It turns out that you can’t order a Dell M3800 laptop with Ubuntu and a second hard drive.

Seriously.

In the configuration I want, I want a 256GB SSD for the primary drive and a 1TB HDD for the secondary drive. That should make the operating system fast while giving me lots of room for files and git repos on the slower HDD.

But when I check it, I get this error:

Choosing Windows makes it go away.

I was dumbfounded. The issue that kept me from getting my laptop, and it appears the issue that will keep me from getting this laptop at all, it that Dell doesn’t know how to deal with a second hard drive on Ubuntu.

Just to make sure, I did one of those chat-thingies:

Time 	                Details
03/06/2015 10:24:51AM 	Session Started with Agent (Jayant K S)
03/06/2015 10:24:51AM 	Tarus Balog: "."
03/06/2015 10:24:59AM 	Agent (Jayant K S): "Welcome to Dell US Small Business Chat! My name is Jayant Kumar Singh and I will be your Dell.com Sales Chat Expert. I can be reached at jayant_k_singh@dell.com or via phone at 1-800-289-3355 ext. 4166817. How may I help you today?"
03/06/2015 10:25:07AM 	Agent (Jayant K S): "Hi Tarus :-)"
03/06/2015 10:25:26AM 	Tarus Balog: "I'm trying to order a Dell M3800 laptop with Ubuntu, but it tells me I can't get a secondary hard drive with Ubuntu, only Windows. Is this true?"
03/06/2015 10:25:52AM 	Agent (Jayant K S): "I am sorry about the inconveinence. Glad you chatted in today, I will try my best to help you"
03/06/2015 10:25:57AM 	Agent (Jayant K S): "Let me check"
03/06/2015 10:27:17AM 	Tarus Balog: "I get that error"
03/06/2015 10:27:26AM 	Tarus Balog: "and it goes away if I choose Windows"
03/06/2015 10:27:37AM 	Agent (Jayant K S): "ok"
03/06/2015 10:29:55AM 	Agent (Jayant K S): "I am working on it please stay connected"
03/06/2015 10:30:14AM 	Tarus Balog: "ok"
03/06/2015 10:32:12AM 	Agent (Jayant K S): "how much boot drive space do you need and how much for the second"
03/06/2015 10:33:24AM 	Tarus Balog: "I was going to order a 256GB SSD for primary and 1TD HDD for secondary"
03/06/2015 10:34:12AM 	Agent (Jayant K S): "Give me 1 Minute"
03/06/2015 10:36:53AM 	Agent (Jayant K S): "I am sorry the second hard drive is not allowed"
03/06/2015 10:37:44AM 	Tarus Balog: "Okay"

My guess is that the Dell provisioning process is so rigid that when it comes to Ubuntu they don’t know how to mount the second drive. This causes the whole thing to fall apart. I don’t know why just mounting it as /data isn’t acceptable, but just when I thought Dell was getting it together when it comes to Linux it appears it is just so much black magic to them.

My hope is that someone from the Dell Linux team will actually see this post and will reply. There is only one thing I really want to know and I have not been able to find out: are there any special PPA’s that ship with the Ubuntu version of the M3800 for drivers, etc. If not, then I’ll buy the Windows version, wipe it and at least have the hardware I want. Yes, it costs me over a $100 more for something I’ll just throw away, but at least I’ll have my laptop issue solved.

This whole process has really soured me on a brand I used to like. My current laptop is the Dell XPS 13 Ubuntu version I bought several years ago and I still like it – I just need more screen real estate. I see now why Apple is able to dominate this market. They always under-promised and over delivered (I never had an Apple order show up late and most showed up a day or more early). I never got some crazy “configuration mismatch” errors when trying to place an order.

And in the few times that Apple made a mistake, they went out of their way to make it better.

A Quick DMCA Note

I just read an interesting article on Google and DMCA takedown notices.

OpenNMS was recently the victim of a bogus takedown notice. While that has been resolved (hats off to Google for dealing with 20 million of these things per month), I thought it funny that, when you typed in “OpenNMS data collection how-to” in a Google search, there was a link at the bottom explaining that one link had been removed as the result of a DMCA action, but it included a link to the original complaint that included the links involved.

So it was still possible to find the link – it just took a little extra work.

This just further illustrates the silliness of this stupid law. Without that link, we wouldn’t have been able to showcase how F5 Networks and their shoddy lawyers improperly included four OpenNMS related links in their takedown notice, so if this new action results in even less transparency, it may become impossible to right such abuses.

If our elected representatives would spend less time worrying about silly things and focus on laws like this that cause real damage to productivity, perhaps the economy would correct itself.

Phone Unlocking

Okay, I just scanned through my Google Reader feed (adios, Reader, I’ll miss ya) and found yet another rant about unlocking phones and I just can’t keep silent. My guess is that this won’t be one of my better posts, so you might as well go and check out xkcd. It’s always a hoot.

In the US at least, when you buy a phone you can often get it at a greatly discounted price if you buy it “locked” to a particular carrier’s network. The debate is whether or not one can legally unlock a phone without the carrier’s permission.

Now, it has always been illegal to unlock a phone while it is “under contract”. This has nothing to do with copyright law and everything to do with contract law. These days most carriers will unlock your phone once it is out of the time span of your agreement. The 3GS I use while overseas was happily unlocked by AT&T since its contract was over.

However, there are people providing software to unlock your phone without the carrier’s permission. This is usually done to circumvent the contract so that one can get a great deal on a phone yet use it on another network. This doesn’t make much sense to do in order to switch to another network full time as you are still under contract to make payments to the original carrier, but it can save a bunch of money if you travel overseas a lot and want to use your same phone yet not get raped by high mobile data fees.

The carriers are trying to use the Digital Millennium Copyright Act (DMCA) – as flawed a piece of legislation as there ever was – to block this software.

Since the DMCA deals with copyright, the administration of the law falls on the Librarian of Congress. They can issue exemptions to the DMCA, and for the last three years unlocking phones has enjoyed this exemption. This year the exemption was not renewed, and so it is now possible for the carriers to pursue legal action against people and companies involved in unlocking phones.

NOTE: This has nothing, zero, nada, zilch to do with “rooting” a device or installing different software on it, which is still legal.

This has caused a bunch of moral outrage, including a petition to get the White House involved.

I think it’s crap.

You want to sign a petition? Sign one to get the DMCA thrown out. When a huge company like F5 Networks can issue a letter to Google that results in the removal of legitimate webpages, like several dealing with the OpenNMS project, and costs me time and effort to try and get it reversed – that should be illegal. Heck, phone unlocking isn’t even a copyright issue so I doubt a court test would hold up, but abuse of the DMCA with junk such as this should be illegal.

You want to sign a petition? Sign one to get carriers to remove the “no class action” clause from their contracts. Since they all do it, it’s collusion, and there is no place in the market one can turn to that allows consumers to have any real recourse when the carriers do something stupid. If the phone unlocking hubbub was about requiring carriers to unlock out of contract phones, I could get behind it, but as far as I know, they do that already.

But if you just want to get out of your legal obligations to save a few bucks, cry me a river. When the Librarian originally granted the exemption, consumers had few options in getting either unlocked phones or getting off-contract phones unlocked. This has changed and the decision to remove the exemption is a good one.

I am especially unhappy when I see people in the Free and Open software movement unlocking their phones early. Enough people already associate free software advocates with software pirates that we need to hold ourselves to a higher standard. Don’t like a contract? Don’t sign it.

But I truly wish we could remove the DMCA and replace it with something less insane. That’s a petition I’ll sign.

Hat tip: This great article on what’s really going on.