2017 Europe: Three SIM card

Just a short post to praise the Three SIM card I bought in the UK several years ago.

I tend to buy unlocked phones and so when I travel I like to get a local SIM card, mainly for data. For this trip this was going to prove difficult, as I’m visiting five countries in nine days.

One thing I like about my Three SIM is that it never gets disabled. As long as I have a balance I have never had a problem, although I do travel enough that I end up using it at least once every six months or so. I am not able to top up the card on the Three website since I don’t have a UK credit card, so I simply use Mobiletopup.co.uk to get a £20 voucher from Paypal. Using that I just buy an “All in One 20” add-on which gives me 12GB of network access, 300 minutes and 3000 SMS messages – way more than I need. I turn that on before I leave the US and my phone works when I land.

What’s wonderful about it is that the plan is valid in any EU country. So far this trip I’ve used it in London, Helsinki and Tallinn, and I expect it to work in Riga and Brussels. I have yet to experience any network issues, although I have not moved far outside of major metropolitan areas.

I have no idea if Brexit will change this plan, but I sincerely hope not. So much of the technology I use in my life comes with headaches that I am grateful when things just work. Thanks Three.

Strong Encryption and Death

I try to use strong encryption wherever I can. While I doubt it will keep my thoughts from prying eyes forever, at least it should make peeking a little harder.

But it dawned on me: what happens when I die? I want to let my business partners see what is on my encrypted desktop and I know my wife will need access to the files on my systems at home. I could share them with her now, but my passphrases are complex and she isn’t very familiar with the operating systems I use.

Now I’m not planning on dying any time soon, in fact I want to live until I am at least 95 and a half. Why that age? Because that is when Halley’s Comet will return. I saw the comet when I was living in California in 1986 and I could care less about seeing it again, but I do want to be the old guy they interview:

“Back in ’86, now that’s 1986 for you young folks, I was livin’ in Los Angeles. The comet was too dim to see in the city, so we drove out to Joshua Tree …”

Halley's Comet 1986

So, how do I safely pass on my important passphrases? This is the solution I chose.

I created a file called “deathnote.txt” which I then encrypted using GPG:

gpg --encrypt --recipient tarus@opennms.com \
    --recipient alice@example.com \
    --recipient bob@example.com deathnote.txt

This will encrypt the file so that both Bob and Alice can read it (and I can too). I then sent it to several friends unrelated to them with instructions that, upon my death (but not before), please send this file to Bob and Alice. I also remembered to include a copy of my GPG private key:

gpg --export-secret-keys -a tarus@opennms.com

Just in case they can’t find it on my systems.

This does require a certain level of trust in my friends, but I am blessed with having several I can count on. As long as I remember to keep it updated this should provide a secure way to pass on this important information, although I hope no one has to use it any time soon.

Move to Let’s Encrypt – it’s soooo easy!

This weekend I wanted to play around with setting up Nextcloud on my home network (we already use it at work and it is awesome). Since I am planning on putting personal information into that app, I wanted to make sure that access to it was encrypted end-to-end.

This meant setting up SSL on my home web server. Now, it used to be that you either had to use a self-signed certificate (which could cause problems) or you had to spend a bunch of money on a certificate from a recognized Certificate Authority (CA).

Enter Let’s Encrypt. Launched in April of this year, Let’s Encrypt provides free certificates that are recognized by most of the things you need to recognize them.

I had been putting it off since dealing with certs is, quite frankly, a pain. You have to fill out a request, send it to the CA, get back a key file, install it in the write place, etc. Even with a free one I didn’t have time for the hassle.

I shouldn’t have worried – with Certbot it is dead simple. Seriously.

Certbot Screen

I went to their site (as directed from the Let’s Encrypt site) and just followed the instructions. I downloaded a script which downloaded all the required dependencies via apt, answered a few questions, and, bam, I had a functioning web server running SSL. They even prompted me if I wanted all requests to port 80 (http) to be redirected to port 443 (https) and when I said “yes” it did it for me.

The whole process took a couple of minutes.

Amazing stuff. The certificates are only good for 90 days, but they even include an automated way to update them.

Certbot Certificate Renewal

As more and more of our personal information becomes digitized, it is extremely important to use strong encryption. In the past this could be inconvenient if not outright difficult, but you really don’t have an excuse with Let’s Encrypt. Use it.

Horizon 16.0.4 Security Release

In response to the Apache Commons library that OpenNMS uses, version 16.0.4 has been released to help secure against a remote exploit.

The exploit involves Java Remote Method Invocation (RMI) which listens on port 1099 by default. In my previous post I pointed out that if that port is inaccessible, then the exploit can’t happen.

What 16.0.4 does is limit RMI to only listen on localhost. While that will prevent remote exploits even in the event port 1099 is blocked via the firewall, it doesn’t completely solve the problem. To fix the root cause of the issue will require changes to Apache Commons, and we are ready to upgrade to the fixed version as soon as it is available.

We tend to be very internally critical of security issues within OpenNMS, and some people complained that my last post wasn’t technical enough. So I’m hoping to correct that with this one, but if you don’t care about such things you should probably skip it (grin). I have started updating the Security Considerations page on the wiki with details about securing OpenNMS in general, and that will have better information for people interested in security and OpenNMS than this blog post.

While blocking external access to port 1099 will secure OpenNMS against this attack for most people, it doesn’t prevent people who have access to the machine from exploiting the vulnerability. This is called a “privilege escalation” attack vs. a “remote exploit”, as a “normal” user can now have rights (i.e. root access) if they are locally on the machine. Most of our users tend to limit shell access to the server, so this shouldn’t be a problem, but in environments that rely heavily on directory services such as LDAP, the default may be to allow non-privileged access to certain users (say, the “IT Group”) that aren’t involved in maintaining OpenNMS.

And there is also the slim chance that there is a vulnerability in our webUI that could allow a user access to the system. We, of course, don’t know of any and we take great care to prevent it, but simply hoping to limit access to the server as a way to prevent this exploit is insufficient.

So, to prevent it entirely, we are removing RMI. It was introduced in the first iteration of the OpenNMS Remote Poller, but real world installation found that getting the proper ports open was a real pain. So instead the remote poller now talks over HTTP/HTTPS (with the latter being the most secure). Most networks have ports 80 and 443 open, so that made things a lot easier.

Until that is introduced (most likely with Horizon 17), it is still a good idea to limit access to the OpenNMS server to only essential people.

Note that Java Management Extensions (JMX) also use serialized objects and thus could be vulnerable. OpenNMS has a JMX port (18980) but it is bound to localhost by default. In fact, all ports are bound to localhost by default in 16.0.4 except for the webUI, port 8980.

There are a number of other steps you can take to harden your OpenNMS server. I’m planning on detailing them on the wiki, but start with only doing a minimal operating system install. The less software on the system, the smaller the chance one will have a vulnerability.

Also, OpenNMS currently runs as the “root” user. This is due to the fact that it needs access to ICMP traffic as well as port 162 for SNMP traps. Both of these require root by default. With some “stupid kernel tricks” you can run OpenNMS as a non-root user, but it has not been heavily tested. We have a detailed list of issues for running as non-root on our Jira instance.

Sorry to drone on about this, but we take security extremely seriously at OpenNMS. We also have to labor under the misconception that Java is inherently unsafe. It is not true, although people still have nightmares from the early issues with client-side Java applets. The Java in OpenNMS is server-side and we don’t use applets, and the language is used securely in a tremendous amount of software.

For comparison, WordPress, an application I love, is currently estimated to run 25% of the world’s websites. It is written in PHP, a language that has a huge track record of security exploits, and many of the spam e-mails I get link to compromised WordPress sites.

It is possible to secure WordPress (we use it for all of our websites as well) but it takes some diligence. We will remain as diligent as we can concerning the security of OpenNMS, and we will continue to take steps to make it even more secure.

OpenNMS RMI Exploit

Recently, my RSS feed on OpenNMS stories turned up an article listing a possible remote code execution exploit in a number of applications, including OpenNMS.

In it, the researcher shows that it is possible to execute code on the OpenNMS server remotely due to a bug in the Apache commons library, which OpenNMS uses.

We’re a little unhappy that they published this without letting us know first (note that the e-mail address “security at opennms dot org” exists for reporting such things), but it is pretty easy to make sure that your instance of OpenNMS is safe. Simply configure the server’s firewall to disable remote access to port 1099 (it will need to remain for localhost).

I was happy to notice that the example he uses seems to be related to OpenNMS running on Windows. It can be a bit tricky to get OpenNMS to work on Windows, and perhaps the Windows default firewall doesn’t block port 1099 so that it why they noticed it.

It is a good idea to run something like iptables on your OpenNMS server and limit remote access to a minimal set of ports. Technically, the only port you really need access to is 8980, which is the default port for the webUI. I would assume that you would want port 22 for ssh access (unless you want to use the console for all configuration). In addition, port 162 should be open for SNMP trap reception.

That should be it. Now the application needs access to other ports (such as 5817 for events) so those need to remain accessible from localhost ( or ::1) but that limits all exposure to only people who have shell access to the server, which we assume you limit to those people you trust. Remember to include IPv6 firewall rules if you use it.

An easy test to see if that port is remotely accessible would be to run:

telnet [IP or hostname of OpenNMS server] 1099

from a remote system to see if you can access the port. No connection should be made.

Sorry about this, but as I mentioned this wasn’t revealed to us until after the exploit was public. We are looking in to how we can better protect against this issue from a code change standpoint, but until then simply blocking access to the port will prevent most problems. We do plan to have a code fix in place soon.

Solution for One Trackpad Issue for the XPS 13

My new laptop is the beautiful new Dell XPS 13 running Ubuntu Gnome 15.04.

It is not perfect, but it is getting close. Lightweight, beautiful screen and awesome battery life (nearly 8 hours the way I use it).

One thing that was killing me, though, was that after a certain amount of time (on the order of tens of minutes and not hours), the trackpad/clickpad thingie would start misbehaving under Gnome Shell, registering bogus clicks. There wasn’t an easy way to fix it outside of a) reboot or b) use an external mouse.

It seems that this issue has been addressed in the 4.1 kernel, so I decided to try it. I’m not sure if Ubuntu is going to support the 4 kernel series officially before 15.10 so I didn’t want to wait.

I downloaded the 4.1.1 kernel here (you’ll need three debs: the “all” headers deb and the image and headers debs for your CPU – I used “generic” and “amd64”), installed them with “sudo dpkg -i” and rebooted. The problem seems to be fixed.

But, my Broadcom wireless driver wouldn’t work. I had to download one more deb from here (via my phone – never play with kernels when you are on a long road trip), install it and now wireless is back.

Now if we could just get palm detection fixed …

OnePlus Class Action?

Ten days ago I did a post about touchscreen issues I’ve been having with my (previously) beloved OnePlus One smartphone. Since then all I’ve experienced from OnePlus customer “care” are delaying tactics and an obvious reluctance to address a systemic problem with their phone design. While I loved this handset while it worked, I won’t be owning another OnePlus product and I encourage my three readers to avoid the company like they would the plague.

I really didn’t expect much from the support process and I wasn’t disappointed. OnePlus has always struck me as a company with great ideas but they’ve always seemed a little over their head when it comes to actually implementing them. But I decided to soldier on and go through the process. I sent in a support ticket on May 11th:

One Plus Support Request 1

The next day (well, about 13 hours later) I got a reply. Not bad, actually, and I developed some false hope that this would work out.

One Plus Support Request 2

So “Kathy” wants me to send in a video. Okay, no worries. I made the video and sent them the link. This seemed to satisfy Kathy who escalated my issue, but then “Leah” also asked for a video.

One Plus Support Request 3

WTF? Okay, definitely a FAIL on reading comprehension, but I replied with a link to the original video and asked them what else they wanted to see. The next message, from Canoy Gem, asks for, you guessed it, another video:

One Plus Support Request 4

At this point it time it has become obvious to me that they are just stalling. There are a number of threads about this issue on their forums (here is the first one and now there is a second – both with pages and pages of comments). So I write back to Gem, again with a link to the video, and he replied with even more requests, this time for pictures:

One Plus Support Request 5

As I’ve seen with the replies from others on their forums, this seems to be pretty common – asking for videos and pictures. I waited until I had some decent light and took really nice pictures of my undamaged phone. However, I was unable to get the back cover off for the final picture. I’ve disassembled a number of devices over the years and while I could probably get this cover off it wouldn’t be without damage. If I damaged it, OnePlus would use it to deny warranty coverage. However, it looks like they are not going to proceed until I do.

One Plus Support Request 6

Note that in this entire exchange they have never mentioned that it might be corrected with a firmware fix (as talked about in the forums). I doubt this is the case with my phone as a) it just started happening and b) it seems restricted to the upper half of the screen, but I would have been willing to test it for them if they’d bring it up.

Also, I’ve noticed that most of the people responding to me have female names. This is a tactic in customer support as women are often treated better in such situations. While they may exist I’m pretty sure OnePlus technical support consists of one overworked guy named Zhang Wei.

I replied that my patience was at an end and either they would let me send them the phone that they could then examine to their heart’s content or I would pursue other actions. All I’ve done for now is replace it with a Nexus 6, but it seems to me that this is a prime example of a use case for a class action lawsuit: A large class of consumers has been apparently defrauded by a vendor supplying faulty products.

I’m talking to friends of mine with some experience in this, but if you have any suggestions for a firm to handle a class action lawsuit, please let me know.

Electronic Program Guide Changes at Schedules Direct

I just noticed that my OpenELEC, Kodi and Tvheadend based DVR was no longer updating the Electronic Program Guide (EPG).

I would get the error:

Service description 'http://docs.tms.tribune.com/tech/tmsdatadirect/schedulesdirect/tvDataDelivery.wsdl' can't be loaded: 500 Can't connect to docs.tms.tribune.com:80 (Connection timed out)

when running the fetch script.

Digging around, I found out the reason is that the Gracenote service is being discontinued and thus some URLs have changed.

I use a script called tv_grab_na_dd from the Debian (wheezy) xmltv-utils package. Version 0.5.63-2 doesn’t appear to use the new URLs. The link above suggests adding:  docs.tms.tribune.com webservices.schedulesdirect.tmsdatadirect.com

to /etc/hosts and that worked well for me. Of course, if the IP address for Schedules Direct ever changes it will need to be updated.

It looks like this is fixed in xmltv-utils version 0.5.66.

Building an Open Source PVR: Step Three – Electronic Program Guide

The most frustrating thing about this project was getting the Electronic Program Guide (EPG) to work. Unfortunately, it isn’t easy.

This was one of the things that TiVo excels at doing. You are basically paying for a very up to date program guide. They also offered something called a “Season Pass” which would cause all of the episodes of a particular program to be recorded without having to explicitly select them.

When I got my EyeTV system, this part was a snap. They partner with TV Guide to provide the service, and unlike TiVo’s $14.95/month fee it is a yearly fee of $19.95 (with the first year being included with the unit).

Even my Sony Bravia is able to get over the air EPG information, but I wasn’t able to get that to work with OpenELEC.

The actual EPG configuration occurs in the Tvheadend software. You get a screen like this:

There are three main areas of configuration: the “Internal Grabber”, “Over-the-air Grabbers” and “External Interfaces”.

The internal section displayed a cron job but no module options. None of the OTA grabbers seemed to work, and there wasn’t a module for North America. That left the external grabbers.

I started digging around and found that it really isn’t easy to get this running.

One tool that kept coming up was XMLTV. On the frontend configuration for the Tvheadend client in OpenELEC they even have a section on it:

XMLTV is a number of things, such as a format for representing TV listings, but it is mainly a set of tools “to obtain, manipulate, and search TV Listings”. It contains programs that will connect to an external source to gather EPG data.

Unfortunately, OpenELEC doesn’t ship with it. There is a script called “tv_grab_file” which is used to manage the XMLTV data, but not to actually acquire that data.

For me the easiest solution was to install XMLTV via apt on my home Debian server. It comes with a script called tv_grab_na_dd that can be used to fetch the data.

But I still wasn’t done. I needed a data source. It looks like all the cool kids use Schedules Direct. They are a non-profit that promotes open source software and provides, for a fee, access to EPG information. Since they had a free trial I signed up, configured my tv_grab_na_dd script to access their information, and voilà, I had an XML file with what appeared to be useful information.

I placed that in the webroot of my server, and then configured OpenELEC to point to it. Nothing happened. So I copied the file to the OpenELEC server, modified the client to use the “FILE” method (see screenshot above) and nothing happened.

I finally had to uncheck the XMLTV checkbox under “External Interfaces”. When I did that I finally had something under the “Internal Grabbers” section.

The last chore was to associate the channels I had discovered with the program guide.

Prior to getting all of that to work, the drop down for “EPG Source” had been blank.

So, to summarize my steps:

  1. Get an account at schedulesdirect.org
  2. Install the XMLTV tools somewhere (I used a Debian box)
  3. Configure XMLTV to access your Schedules Direct account
  4. Set up a cron job to periodically grab the updated EPG information and store it in a web root:
     0 1,13 * * * /usr/bin/tv_grab_na_dd --config-file ~/xmltv.conf --days 7 > /secure/html/xmltv.xml
  5. On the OpenELEC box, set up a cron to fetch the data:
    0 2,14 * * * wget -O /storage/xmltv.xml

Whew. So far everything has been working well. You want to be sure not to fetch the data too often as that can overwhelm the Schedules Direct servers. My current seven day XML file is about 10MB.

I went ahead and signed up for a year account for $25, bringing my total to $705.92 (the hardware was $680.92 and the software was, yup, $0). It’s quite possible to shave off about $200 by going with less memory and a smaller SSD (or using an HDD) or if you already have a server to run the Tvheadend backend you could get by with a Raspberry Pi.

My next steps are to play with all the cool add-ons and to try and organize my pictures in a fashion where they would be usable with the system. More fun for me.