Planet OpenNMS

June 26, 2017

This Week in OpenNMS: June 26th, 2017

It's time for This Week in OpenNMS! In the last week we did various internal and web UI work.

Github Project Updates

  • Internals, APIs, and Documentation

    • Ron Roskens added documentation about collection resource types.
    • Brynjar Eide fixed a bug in scanning SNMP devices with missing IP-MIB::ipAddressIfIndex entries.
    • Jesse did more work on Java 9 support.
    • Antonio squashed more topology and Enlinkd bugs.
    • Jean-Marie Kubek improved test support for RADIUS by using an internal tinyRa...

June 26, 2017 03:23 PM

June 21, 2017

A Brief History of an Open Source Company

I’ve been invited to give a keynote at this year’s Ohio Linuxfest being held in Columbus, Ohio, on 29-30 September. I am both excited and humbled as this is one of my favorite conferences of the year and I know a lot of amazing people will be there to share their knowledge of free and open source software.

Ohio Linuxfest Logo

I take my presentations pretty seriously, especially keynotes, so I wanted to come up with something that was both funny and interesting. They asked me to speak on running a business around open source software, and I immediately thought I should come up with some click-bait title like “Ten Things About Open Source Business, Four of Them Will Shock You!” but it just didn’t feel right. Then I thought about Hawking’s A Brief History of Time and that seemed more fitting.

My most popular talk so far has been on starting an open source business, but that focuses mainly on the mechanics of the process. For this talk I want to trace my history with OpenNMS starting with my first day on the job and then describing how it grew to become what it is today. In those 15+ years I’ve had a lot of adventures, some good and some bad, and I’ve met a some wonderful people. It is the work of many of those people that actually make OpenNMS what it is – I act more like a “crap umbrella” with my one job being to block all of the things that might keep the team from being productive – and I want to talk how that came about. This presentation will consist almost entirely of real world examples of the problems we encountered and our decision process for solving them.

I hope it will be entertaining and useful, and look forward to seeing you there.

by Tarus at June 21, 2017 06:26 PM

June 20, 2017

When Not To Start an Open Source Company

Over the weekend, Chris Aniszczyk posted a link on Twitter to a very interesting article by Matt Klein about his decision not to start an open source company around his project, Envoy. I thought it raised a number of interesting points worth a few comments.

First off, Matt works for Lyft, which, in case you haven’t heard of it, is Uber without the moral decay. I abandoned Uber some time ago, despite being an early adopter, and I’ve been very happy with Lyft. One of the main differences is that Lyft allows you to tip your driver, which I almost always do with few exceptions. The fact that Lyft is able to keep and motivate people like Matt speaks volumes for their corporate culture.

It also demonstrates a wonderful trend of commercial companies starting and maintaining open source projects. I’ve been working with open source for almost two decades and I can remember when any software developed at a company was considered confidential. To this day there are a number of vendors who consider their SNMP MIB files (which, I should point out, are really only useful to people who have purchased their products) proprietary information. Companies like Lyft, Paypal and Facebook, none of which would self-identify as open source companies, have gained a lot of value for little cost by making the tools they use open source.

When talking about open source for the enterprise, I often talk about the fact that it is the processes that a company uses to serve its customers that make it unique and define its value, not the tools used by the company. So often with commercial software you have to change those process to fit how the application thinks you should work, and in the process you lose some part of what makes you special to your customers. With open source you can fit the application to those processes. It is how you use the tools and not the tools themselves that is important, and so there is a lot to gain and little to lose by making them open source.

Getting back to Matt’s article, he is a project maintainer for Envoy, which is a “high performance C++ distributed proxy and communication bus designed for large service oriented architectures.” While I don’t consider myself a coder so I don’t claim to fully understand the its advantages, I do recognize enough buzzwords in that sentence to know that it would attract some attention from investors, and Matt was approached about leaving Lyft to start a commercial business around Envoy. He decided not to, and as I read his article about his decision I realized I’d found a kindred soul, someone who was more interested in creating something of value that would last versus making a quick buck.

He had me with this paragraph:

In my opinion, the best opportunity to commercialize OSS lies with projects that can be easily turned into SaaS products. Ultimately, even if software is completely open, many customers are happy to pay for a turnkey solution that “just works” and has a defined SLA with 24/7 monitoring and support. In some sense, customers pay for the operational expertise that comes from deeply understanding and running the software, versus the software itself.

Amen.

I’ve been making a living on open source for 15 years now working with OpenNMS, and I’ve spent a lot of time thinking about business models. We started out with the “service and support” model, which kept the doors open but limited growth. Then our clients started asking us for features, so we added custom development, which was time intensive but allowed us to finance OpenNMS features which attracted even more customers as the platform became more powerful. When we hit the problem of trying to balance the “release early, release often” philosophy of open source with the need for stability, we adopted the Red Hat model of splitting our application into a feature-rich, rapidly developed release (which we call Horizon™, similar to Fedora) and a more stable, subscription-based release that may lag in features but is better suited production environments (which we call Meridian®, similar to RHEL). But ultimately we came to the decision that what we really wanted to do was to offer OpenNMS as a service.

One company that inspired that decision was Automattic, maintainers of WordPress. I don’t think I know of a more powerful piece of software that is easier to install. They have a famous “5 Minute Install” that is quite simple. First, you drop the software into the webroot of your web server of choice. Next, you create a database account on your database of choice with certain permissions. Then you navigate to a web page and follow the prompts.

However, for a lot of people, terms like “webroot” are gibberish, and even with WordPress you still need some minimal database skills to maintain it. So Automattic offers up WordPress as a service. For a small monthly fee they’ll do everything for you, and this has generated revenues on the order of tens of millions of dollars per year.

OpenNMS is way more complicated, thus the value of a hosted version should be greater. In order to do so we needed some way to access the client’s network in a secure fashion, so with Horizon 20 we introduced the Minion. The Minion software allows for OpenNMS functionality to be distributed. It is built on the Karaf container, so once installed all of its features can be remotely managed. For smaller networks, the Minion can be sold as an appliance and talk to a hosted version of OpenNMS. It can bring a complex and powerful tool like OpenNMS into the hands of the masses.

For larger companies it solves issues of scale as Minions can be deployed to cover even the largest networks (our goal is IoT scale). We’ve had them in production at one client for months now handling over 2 million events an hour. That translates to around 555 events per second, although the system itself can handle over 10,000 events per second so they have room to grow. If they ever hit that limit, we can simply add more Minions. They have the option of hosting all of OpenNMS in their own data center, or they could choose a hybrid model where some of the functionality is outsourced.

For pretty much the first time in the history of OpenNMS, we are seriously and actively seeking investment. There are a number of companies entering this space who have raised enormous amounts of money, and we think we can be competitive for far less money and provide a better solution. Plus, also for the first time in the history of OpenNMS, we have a reason to make it easier to use versus spending all of our resources making it more powerful.

Matt talks about investment in his post (remember Matt? As usual, I’ve made this all about me. Meeee!) It was actually his stories about dealing with investors that prompted me to write this. As Envoy started to get some traction, investors wanted him to leave and start a company. He writes:

Over the last few months I’ve been told by several investors that no OSS has become ubiquitous without having explicit commercial backing. I think this is false and is situation dependent. If anything, I would argue that if I were to leave Lyft now and start a platform company around Envoy, it will decrease the chance of Envoy becoming ubiquitous, primarily because it would negate all of the reasons laid out above.

That first sentence is interesting, since “ubiquitous” and “commercial” are a little vague. I would make the claim that the Apache web server was ubiquitous until its success spawned NGINX, and it was backed by the Apache Software Foundation which is a non-profit. Is a foundation “commercial”? The idea that for a project to become successful it needs a number of people to spend a lot of time working on it seems obvious, and the best way to achieve that is to pay those people to work on it.

He goes on to write:

It took me a lot of time to ultimately understand the previous simple point. Investors are extremely persuasive. They capitalize on “fear of missing out.” However, it’s important to realize that the opportunity cost is hugely mismatched between investor and company.

When he writes “investors” above I believe he means specifically venture capitalists. We’ve talked with a few VCs in the past and I can remember the almost “strong arm” tactics they used. If I hear “a rising tide lifts all boats” one more time, I might have to hit somebody. I’m not saying that all VCs are the same, but many of them come across as gamblers and not investors. I’m risk friendly but I don’t gamble. I’m heavily invested in wanting to build something with OpenNMS that outlasts me (it is already much bigger than me as the team I work with has way more to do with its success than I do) and I don’t want to gamble with it.

I do hope that there are some investors out there that can appreciate that aspect of our company as well as the fact that we’re profitable, have mature products and wonderful customers. Perhaps private equity or perhaps another company that shares our vision and wants to advance the project through acquisition. In any case we’re looking for them.

When I was a young man, old guys like I am now would tell me “work on something you love, not just for the money”. I always dismissed it with the thought that with enough money I can buy love. When you immerse yourself in something as personal as an open source project for ten to twelve hours a day, year after year, you really do have to love it and the satisfaction you get just can’t be bought. Matt’s thoughts are similar:

Ultimately, on a personal level I’m just having too much fun solving tough computer science problems at large scale at Lyft and building a community around Envoy. The bar to do something different is therefore extremely high, and it took a long time to realize that it’s perfectly OK to accept that and keep going down the existing path that I’m on. On another level, leaving now to start a company would feel very much like not following through on my original goal of open sourcing Envoy; the industry desperately needs a high quality and community-driven solution to microservice networking. Follow-through is something I take very seriously.

With that attitude the success of Envoy is almost assured.

by Tarus at June 20, 2017 03:10 PM

What the...? SNMP Daemon spams my log!

As soon you provide more than a handful of services monitoring and centralized logging becomes essential. Especially when you run the Net-SNMP daemon with Docker you will notice your logs get spamed with some messages like Cannot statfs /run/docker/netns/...: Permission denied everytime you query for performance data.

The Wiki article is a small guide how to configure the Net-SNMP daemon to get cleaner logs and less noise for a cleaner signal.

Would you like to know more? -> SNMP spams my...

June 20, 2017 01:48 AM

June 19, 2017

This Week in OpenNMS: June 19th, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internal and web UI changes.

Github Project Updates

  • Internals, APIs, and Documentation

    • Antonio worked on more Enlinkd and topology bugs.
    • Jesse fixed a bug in Collectd in Horizon 20 that would cause large amounts of collection on non-primary interfaces.
    • I fixed some issues related to the JAXB update.
    • I continued my work on a JavaScript API for OpenNMS.
  • Web

    • Seth worked on the ReSTv2 APIs and cleanu...

June 19, 2017 02:33 PM

June 15, 2017

CubaConf 2017

UPDATE: Today the United States administration announced tougher restrictions on travel to Cuba. While nothing has changed at the moment, there will be some changes in the next 30 days. This should not impact people attempting to go to Cuba for this conference as it should fall under the “professional” or “educational” travel categories. This may change again before November and I’ll be sure to post updates.

While tourist travel remained officially banned, Obama also allowed a broad category of “people to people” visits to Cuba. Trump’s new directive still allows individual travel in all but that category, and reverts to an earlier policy of requiring “people to people” visits only in a Treasury-licensed group.

Free and open source software is as close to a true meritocracy than anything else I’ve found. It doesn’t matter what is the color of your skin, your gender or where you live, your value is judged simply by your contributions to the project. I wrote up my favorite instance of that for opensource.com concerning my friend Alejandro who got involved with OpenNMS when he lived in Venezuela. He and his wife are now permanent residents in the US due to his work on our project.

I actually forget how I came across CubaConf, but I was immediately interested in attending. This is an annual free software conference held in Havana, Cuba.

CubaConf

It has been illegal for US Citizens to travel to Cuba since before I was born. Last year the Obama administration eased some of those restrictions, so it is now possible, under certain conditions, to travel to Cuba as well as to use US Dollars while there.

Cuba has been pretty isolated since the 1960s, and as it races to catch up with the rest of the world it will need access to modern technology, especially software. I see an opportunity for free software to play a huge role in the future of that country, and I am eager to meet the people who will help make that happen.

I want to use this post to encourage all of my free and open source software friends to come to CubaConf. This is a three-day event that follows a format similar to one we used for our OpenNMS user conferences. The first day is a normal conference, with various tracks and presentations set to a schedule. The second day is a “barcamp” style conference where the attendees will set the agenda, and the third day is a hackathon.

Presentations are welcome in both Spanish and English, so I’ve submitted two talks (both in English). One is on starting an open source business. This will be different from my usual talk as I want to focus on how someone in Cuba could both spread the use of free software while getting paid to do it, without as much focus on setting up a corporation or other formal business entity. The second talk is on OpenNMS. While business transactions are still difficult between the US and Cuba, I really want to bring the magic that is OpenNMS to their attention so that when things ease between our countries people will be familiar with it.

I plan to attend all three days, and Alejandro is coming with me to help with any language issues (my Spanish is passable but not nearly as fluent as a native speaker). Note that the Call for Papers is open until the end of August.

Since you might be hesitant to consider going to Cuba from the United States, I wanted to share with you how it works.

First, tourism to Cuba for Americans is still illegal. However, the State Department has come up with a list of 12 categories which qualify for visiting.

12 Visa Categories for Cuba

In the case of CubaConf, you will choose either number four “Professional research and professional meetings” or number five “Educational activities”. I guess number six might work “Public performances, clinics, workshops, athletic and other competitions, and exhibitions” since it is kind of a workshop, but I’d stick with the first two. Since I am a free software professional, I plan to use number four, as I consider this a professional meeting.

Note that Cuba could care less about why you are there – this is a requirement of the US government.

Second, once you have a legal travel category, you’ll need a visa. In speaking with my favorite airline, American Airlines (they offer direct flights to Havana from Charlotte, NC, and Miami, FL), once you book your travel they will outsource the visa process to Cuba Travel Services who will handle the whole thing via e-mail. The visa costs $50 and it looks like there may be a $35 fee, but I’m not sure if the fee applies if you are referred via the airline and it may be built into the price of the ticket.

Speaking of things included in the price, the third thing to consider is that all Americans traveling to Cuba must have non-US health insurance. This is included as a $25 charge when you purchase your ticket.

That covers much of the “getting there” part. The fourth, and in my mind most important thing to know is that Cuba is still very much a cash-only country. American banks are still not doing business there so your credit cards won’t work, nor will the ATM, so you’ll need to bring cash. I verified this with calls to Bank of America, Chase and Citi – currently none of those banks have cards that work in that country.

There are two types of currency in use: The Cuban National Peso (CUP) and the Cuban Convertible Peso (CUC) or “kook”. The CUC is pegged to the dollar and is the currency used by most visitors. Luckily, Havana is a pretty safe place, although I still won’t want to carry around a lot of money if I can avoid it.

I’m not sure where I will stay. Being a big Marriott fan I do have the option to stay at the Four Points Sheraton, but it seems to be pretty far away from the Colegio Universitario San Gerónimo where the conference will be held. Most people visiting stay in a “casa particular” which is a room in someone’s house, and it appears that Airbnb is also in Cuba.

I plan to use the open source way and just ask my friends organizing the conference where I should stay. It is very easy to do, as they have set up a Telegram channel for the conference. While Spanish is the main language in the channel, English is welcome, and if you are thinking about coming to CubaConf I would consider going there first.

I am very exited about the opportunity to visit Havana in November. Despite the modern history between the US and Cuba, I know I’ll make some new friends.

Software libre crea amistades inmediatas.

by Tarus at June 15, 2017 07:22 PM

June 13, 2017

This Week in OpenNMS: June 13th, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internal and web UI changes.

Github Project Updates

  • Internals, APIs, and Documentation

    • Antonio worked on more Enlinkd and topology bugs.
    • Jesse fixed an issue with thresholding WS-Man collection results.
    • Jesse worked on fixing StrafePing on the Minion.
    • Jesse did more work on initial Java 9 support.
    • Seth did more work on upgrading our embedded Karaf to 4.1 and our ActiveMQ to 5.14.
    • Dustin updated the doc...

June 13, 2017 02:59 PM

June 12, 2017

Monitoring Websites with Selenium

Now and then websites are really important for daily business and private life. You can nearly buy everything on websites, find information, meet friends, watch movies and much more. All the worse if your website is not working and you don't notice it or your customer first. But even if you already have some monitoring for webservices or domain name availability, you can't be sure, if your website itself works correct. Maybe the login/logout procedure is broken, some parts of the website are mis...

June 12, 2017 01:42 PM

June 09, 2017

Horizon™ Version 20 Released

Just a heads up that version 20 of Horizon has been released.

Since version 20 coincides with the 20th anniversary of the film The Fifth Element, we’ve decided to use characters from that movie as codenames for this release. Version 20.0.0 is called “Leeloo”.

This release continues our commitment to rapid releases in the Horizon product line, and is mainly focused on bug fixes, small enhancements and code cleanup. We have removed all use of Castor for the parsing of XML files and replaced it with JAXB, and number of deprecated events have been removed from the system.

Probably the biggest new feature is a topology provider that can be used to create custom maps. The Asset Topology Provider generates a GraphML topology based on node metadata including asset fields.

You can read the announcement and for more information, check out the release notes.

by Tarus at June 09, 2017 06:41 PM

June 08, 2017

OpenNMS Horizon 20.0.0

OpenNMS Horizon 20.0.0 (code name: Leeloo) is now available!

This release is focused more on fixes and tuning of existing subsystems, as well as plenty of code cleanup and updates in the process of bringing Minion monitoring up to the level of the rest of the codebase.

New in OpenNMS Horizon 20

Breaking Changes
  • NMS-9303: Deprecated Capsd and Linkd event definitions are removed. Otherwise unused event definitions are also cleaned up and removed from the code and configurations. For a d...

June 08, 2017 07:14 PM

Why the FCC’s Title II is so Important (Spectrum Rant)

Here is a rant about Time Warner/Charter/Spectrum or whatever the heck they call themselves these days. It illustrates how this large company can have a huge negative impact on a small business, and why treating Internet providers as common carriers is so important.

Our company wouldn’t exist without the Internet. Outside of the fact that our products are mainly used to monitor Internet resources, we host a number of servers from our office and about half of the staff works remotely so we rely on the Internet to communicate and coordinate.

Back in 2012 I contracted with Time Warner to provide Internet access to our office. We had fiber to the building and while our service was considerably more expensive than coax, I liked the fact that it was symmetrical and expandable. We started of with 20 Mbps but soon increased that to 50 Mbps. Over five years we only had one outage, due to a misconfiguration of our Customer Premise Equipment (CPE), and they corrected it within 20 minutes. I love the fact that when you called in the person who answered the phone understood terms like “duplex” and they were always very helpful.

Note the scenario: happy customer who is happy paying a premium for enterprise-level service.

Now let me tell you why all that goodwill has gone away.

Earlier this year we decided to move our office from Pittsboro, NC to Apex, NC. The first thing I did was contact Time Warner (well, Charter at the time) to insure that they could provide fiber to the new location. They said they could, although it would take 45 to 60 days. As our new office space needed to be completed, we were targeting an April 1st move in date anyway, so on February 15th I placed the order for the new service. At best, it would be available on the 1st and at worst it would be ready by the 15th. We told the old landlord we’d be out by April 30th just in case and to give us more time to move.

Finally, Spectrum doubled our speed and cut the price in half. I was feeling pretty good about the whole thing.

The feeling didn’t last.

As we got closer to April, things started to go wrong, most of it due to the fact that Spectrum is now such a behemoth that they have no idea what they are doing. In order to get fiber into our new building, they needed what is called a “Right of Entry”. They sent it to our landlord who promptly completed the form and sent it back. However, that person didn’t let the project manager know the form had been received, so he did absolutely nothing. Ten days (!) later I get a note that our build out had been suspended because of the lack of the ROE form. A form, I should point out, that was sent to them, twice.

At the end of March I’m told that our new date is May 11th. I’m unhappy – due to their poor processes I now have a new office that I can’t use for six weeks (remember, we took possession and started paying rent on April 1st). We also had to be out of the old office by the end of April. Luckily I work with a great team that is able to be productive when working from home, so I decided to suck it up and live with it.

On April 12th I get an update – the new date for the end of construction is now May 15th due to processes within Spectrum taking too long to finalize the work with a contractor. Now the actual date we’ll have Internet has been pushed out to the week of May 29th.

I am livid. By this point I’m ready to switch to the other option, AT&T. Unfortunately, they also need 45 to 60 days for service installation so I realize at this point I’m stuck with Spectrum.

I ask my salesperson for options and he suggests we get coax installed for a month (for a fee, of course). Since our office is right next to a large housing development they can get coax in the following week. I sign off on it.

It didn’t happen. When May arrived some of us started working in the new office mooching off the neighbor’s Wi-Fi from AT&T (with permission of course). I ended up traveling for a couple of weeks so I completely forgot about the coax option (it’s not like Spectrum was keeping me updated on anything – I’d have to reach out to them for an update). I did get a note on May 10th that all construction had been completed for the fiber and another note on May 18th that our new install date was June 2nd.

(sigh)

So, 45 days late, we have a firm install date. Wonderful.

Imagine how I felt when on the 24th of May I received a note that more construction was needed and that it would be pushed out another 30 days at least. When I get extremely angry I refer to it as going “non-linear” as that how fast my blood pressure rises. As I was ranting to pretty much everyone I’d ever interacted with at Spectrum it dawned on me that this could be for the coax order. Turns out that was the case. Apparently our crack project manager on the coax side decided to route our service from a point several miles away instead of from the one nearly across the street. This is why it was delayed and why the construction was needed. By this time we are about a week out from having fiber so I canceled the order. I did get a very apologetic call from the coax salesperson which I appreciated (under Spectrum, fiber [Enterprise] is handled by one sales team and coax [Business] is handled by another), and I made it clear that I’d be okay with everything as long as the fiber was delivered as promised on the 2nd.

It was. Around noon on June 2nd we had our 100 Mbps service and on the 3rd we moved all of our devices from the old office in Pittsboro to the new one in Apex. I informed my salesperson that they could disconnect the old service and despite all of the problems, I was happy with the new service.

So the whole process cost me two months rent and a few years off my life, but it was finally over.

Not so fast – the other shoe fell today.

I get an e-mail that I need to confirm my disconnect request. That didn’t bother me, in fact I appreciated it, but what did bother me was an additional note that it would be done within 30 days. When I replied I asked for clarification – would I be *paying* for the service I wasn’t using until they could disconnect it? The answer was “yes”.

I experienced a new word – apoplectic.

Due to the fact that the bureaucracy behind the new merged Spectrum company is so bad, I’m out nearly ten thousand dollars. That is the real money – it’s probably cost us twice that again in lost productivity from lack of network access and dealing with them throughout this process. We’re not one of those companies that is too big to fail so this really impacts us negatively. Had it been explained to me that I’d have to pay for the service until it was disconnected, I would have put the disconnect order in a month ago, but then had I used the date I was originally promised, our servers would have been off-line for over a month. That would have been catastrophic to our company.

Finally, I’ve gone from a happy customer to an extremely pissed off one who will be actively looking for options. Based on my experience I would suggest any business looking for network access look elsewhere.

Access to the Internet has become as important as other utilities such as electricity, water and sewer and just like those utilities it needs to be regulated as one. This is why the decision by the new industry-picked head of the FCC to reverse the decision to classify Internet access under Title II as a “common carrier” is so devastating to businesses like mine. Our company is small, yet we put millions of dollars into the local economy each year. You multiply that by the number of other small businesses and it can have a great impact to any community. Barriers put up by companies like Spectrum demonstrate that they can’t self-regulate and the government needs to take a firmer hand (and this is coming from a left-leaning libertarian).

I will be protesting that final bill for Internet access and I would welcome any advice on how to deal with a company like Spectrum. Let’s hope that there is a change soon so that other businesses can focus on creating value and not have to deal with the crap we had to endure.

I’m not holding my breath.

by Tarus at June 08, 2017 06:47 PM

June 06, 2017

OpenNMS Training Dates Announced for September 2017

The next OpenNMS training course will be held the week of 11 September, 2017, at the OpenNMS headquarters in Apex, NC, USA.

This will be the first training course in our new office. Did I mention we are just above a brewery?

by Tarus at June 06, 2017 02:39 PM

June 05, 2017

This Week in OpenNMS: June 5th, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internal and web UI changes.

Github Project Updates

  • Internals, APIs, and Documentation

    • Jesse fixed an issue with WS-Man thresholding.
    • Jesse fixed an issue with the HikariCP strategy failing to get new connections with a busy database.
    • Jesse did some initial work on fixing Java9 builds.
    • Antonio worked on some Enlinkd and topology bugs.
    • I worked on creating a JavaScript API for interacting with OpenNMS ser...

June 05, 2017 02:59 PM

June 02, 2017

Service Outage Tomorrow, Saturday June 3rd

Wonder of wonders, Time Warner/Charter/Spectrum/whatever has finally delivered connectivity to our new office, albeit a month late.

So, we’ll be moving a number of servers from our old location to the new one, which means certain things, such as demo and Bamboo will be down for a few hours. Almost everything else is hosted elsewhere and redundant, so we shouldn’t have any other issues.

Sorry for the outage and thanks for your patience.

by Tarus at June 02, 2017 10:03 PM

June 01, 2017

Monitoring? Meh.

Recently, I was talking to a person in the tech industry and describing all of the cool things we are doing with OpenNMS, when he kind of cut me off and went “Oh, monitoring? Meh.”

Well, I can’t remember if there was an actual “meh” but that’s how it came across, and I’m afraid the reaction is probably more common that I would think. Monitoring isn’t sexy, but it surprises me that people can’t see how critical it will be to the future of any business.

IoT Devices Over Time

While forecasts vary, by 2020 there are expected to be over 30 billion devices on the Internet, and that figure will skyrocket to over 75 billion by 2025. Just knowing what is connected to your business network is going to become critical, as well as making sure it belongs there in the first place and, if so, is functioning properly.

Outside of the obvious security concerns, as people began to transact business more and more through devices rather than people, faults in those devices will directly impact revenue as people search for other options when faced with a bad experience.

Here are a couple of examples.

One of the greatest inventions in my lifetime is the ability to buy fuel at the pump. You just pull up, swipe your card, pump and then leave. You used to have to pay inside, and some places made you pay first which meant two trips in if you were paying by credit card. It could be cold or rainy, and not only did you have to wait in line behind people buying food or lottery tickets, you had to leave your car out by the pump possibly blocking the next customer.

The only problem I’ve experienced with this process concerns the receipt. Quite frequently I need a receipt, but it seems the pumps I choose are always out of paper. The little red indicator mark when the paper roll is almost finished isn’t visible to the cashier since there really isn’t one out by the pump. It is frustrating, but it is not like I have a choice at the moment. If there was some way to monitor the pump for a “low paper” alarm, it would improve my shopping experience.

One shopping experience that did result in my leaving the store without a purchase happened yesterday at a Lowe’s Home Improvement store. I needed some florescent lights for the new office so I went by on my way home. I picked up four bulbs (two that I needed and two spares) and went to the checkout area.

I walked past several unmanned cash registers until I got to the “Self Checkout” section, which was the only thing open. Of the four machines, two had red blinking lights on them (that are green when things are functioning normally) and the one lone, overworked cashier was doing her best to help people out. I usually don’t mind using Self Checkout and when I noticed one of the two machines was open (everyone else was waiting for the attention of the lone cashier) I went to it and started my purchase.

I scanned my “My Lowe’s” card and then the first bulb. “Eight ninety-five” piped up the voice and I placed it in a bag.

Here is where the problems started. First, I hate the fact that with these Self Checkout kiosks they don’t trust you to use a “quantity” key. I was buying four identical items but I was required to scan each one. Next, the bulb was light enough that it didn’t register as having been bagged, so the interface yelled at me and presented me with a button marked “Skip Bagging Item?”.

I sighed and, having no other option, hit the button. I then went on to scan the next three bulbs. However, as I bagged the fourth bulb, the scale must have started working since the whole unit went into some kind of alarm mode, screeching “Unidentified Object in the Bagging Area!” and the screen was locked until the cashier had time to come and fix it.

I looked around the area, and by this time all four kiosks had a flashing red light, there were at least three shoppers lined up to use them in addition to those of us already there, and our valiant cashier was busy helping a guy ring up his plumbing supply purchase which consisted of a ton of small copper fittings which most likely wouldn’t be registered by the scale.

I gave up. I picked up my bulbs and returned them to the Lighting section, passing three employees in the customer service area helping zero customers. Before I reached the car I’d ordered the same bulbs on Amazon at a fraction of the price, and they’ll be here on Friday.

Yes, I’m complaining, but how could monitoring have helped here? First, there is some sort of monitoring – those little red lights. When they all light up you would assume someone, or perhaps multiple someones, would come by to help. A monitoring system could have made sure that happened by using an additional notification system outside of the lights, and escalating it until the problem was addressed.

A more long term solution would be to collect information on the purchasing experience and the problems people encountered and to make changes to the automated kiosk software. I’m certain that Lowe’s didn’t write that software but instead bought it, and like most proprietary software solutions they now have to fit their processes to the application instead of the other way around. It probably wasn’t designed for a store that sells a lot of small, light things which is central to the issues I have using it.

With the rise of IoT devices, robotics and other forms of automation, monitoring is going to become extremely important. Lowe’s lost out on a $40 sale, but think of something like an assembly line where a problem could result in the loss of thousands of dollars a minute. Our goal at OpenNMS is to be ready for it, and to build products that make people go “Monitoring? Oh yeah!”.

by Tarus at June 01, 2017 12:17 PM

May 30, 2017

This Week in OpenNMS: May 30th, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internal changes, the Minion, and web code.

Github Project Updates

  • Internals

    • Seth continued working on modernizing our bundled ActiveMQ, Camel, and Karaf versions.
    • Jesse added an event for knowing when alarms are deleted (and fixed BSM to track them).
    • Antonio worked on more bug fixes to Enlinkd.
    • Jesse added support for doing reverse DNS lookups even when a A record does not exist.
  • Minion

    • I added su...

May 30, 2017 04:22 PM

May 25, 2017

Server Room Nightmares

I’m interested in any server room nightmares people would like to share.

Here’s one of mine.

We are in the process of moving offices from Pittsboro, NC down the road to Apex. Unfortunately, we are having some issues getting Spectrum Enterprise to complete the fiber installation at the new place, so while we are out of our old building the lack of network access in the new building means we have a bunch of servers in the old location.

Today while I was working in the new office and mooching of our kind neighbor’s wi-fi, I got several notices that links had failed.

linkDown event list

These were some workstations that we use for training, but when they are not in use we use them as part of our continuous improvement Bamboo farm. I immediately hopped on our Mattermost IT channel and asked if anyone was rebooting or otherwise messing with the machines, and when the answer was “no” I started to investigate.

One suggestion was that the air conditioning may have failed and those machines shut down from overheating. It has happened in the past, but it was both rather cool today and other machines that are more sensitive to such things were still running. I checked it out anyway using our AKCP probe.

temperature graph

The temperature had increased a bit, but it wasn’t anything that should have caused problems (it was caused by the server room door being left open).

Being 30 minutes away, I decided to text my friend Donnie, who is technically gifted as well as working in our old location, and he went to investigate.

For some reason, those three machines had been disconnected from the switch.

Now just for this situation we have an Arlo camera installed in the server room, so using the time stamp on the linkDown traps I found the following video.

Note the slightly balding guy in the red shirt in the lower left corner of the video. He is busy unplugging our devices.

Why? I have no idea. These people represent the IT people for the new tenant, and I assume they had legitimate reasons for being in the server room but messing with our equipment was not one of them.

Seriously, in over 30 years of working with computers, I’ve never heard of anyone going into someone’s house, office, server room or data center and just start unplugging cables. I still have not heard an explanation, but the landlord has had a discussion with the new tenant and it shouldn’t be happening again. It is one reason the important stuff is in that locked half-rack seen in the upper left corner of the video, and the really important stuff is hosted elsewhere.

I am curious – I’m certain this pales compared to other stories out there. Do you have any whoppers to share?

by Tarus at May 25, 2017 07:23 PM

May 23, 2017

New Meridian® Releases Available

Just a quick note to point out that new Meridian releases are now available: 2015.1.5 and 2016.1.5

For those who aren’t aware, Meridian is a subscription-based version of OpenNMS built to complement Horizon, the cutting edge release. You can think of it as Meridian is our Red Hat Enterprise Linux to Horizon’s Fedora. There is one major Meridian release per year and each major release is supported for three years.

Before the Meridian/Horizon split it was taking us 18 months or so to do a new major release of OpenNMS. Now we do three to four Horizon major releases a year.

About half of our revenue comes from support contracts and so we had to be extra careful when doing a release, and even with that many of our customers were reluctant to upgrade because the process could be involved. This was bad for two main reasons: often they wouldn’t get bug fixes which meant an increase in support tickets, and more importantly they might miss security updates.

Updates to Meridian, within a major release, are dead simple. This is the process I used yesterday to upgrade our production instance of OpenNMS.

First, I made a backup of the /opt/opennms/etc and /opt/opennms/jetty-webapps/opennms directories. The first is out of habit since configuration files shouldn’t change between point releases, but the second is to preserve any customizations made to the webUI. I modify the main OpenNMS page to include a “weather widget” and that customization gets removed on upgrades. Most users won’t have an issue but just in case I like having a backup.

Next, I stop OpenNMS and run yum install opennms which will download and install the new release. The final step is to run /opt/opennms/bin/install -dis to insure the database is up to date.

And that’s it. In my case, I copy the index.jsp from my backup to restore the weather information, but otherwise you just restart OpenNMS. The process takes minutes and is basically as fast as your Internet connection.

If you have a Meridian subscription, be sure to upgrade as soon as you are able, and if you don’t, what are you waiting for? (grin)

by Tarus at May 23, 2017 03:28 PM

May 22, 2017

OpenNMS Meridian 2016.1.5 Released

Release 2016.1.5 is an update to 2016.1.4 that provides quite a few bug fixes and a few enhancements.

The codename for 2016.1.4 is Peirce Quincuncial.

Bug

  • OpenNMS doesn’t correctly support IPv6 addresses for (jasper) reporting (Issue NMS-5450
  • vmware urls do not support username/passwords that require URL encoding (Issue )(NMS-7106)
  • If a target node is rebooted the RRD/JRB files contains spikes because the sysUpTime check is not working (Issue )(NMS-7148)
  • Geo-Maps running on a server without internet connection breaks the UI for valid nodes. (Issue http://issues.opennms.org/browse/NMS-7148))
  • scrollbars displayed for RrdDashlet title box (Issue NMS-7562)
  • RWS client always uses plain HTTP, even if rws-config.html has HTTPS URL (Issue NMS-8441)
  • Multiple smoke tests flapping inside docker (Issue NMS-8544)
  • Restarting OMNS results in numerous SNMP outage alarms and notifications (Issue NMS-8620)
  • Null ospfrouterid in NodeDiscoveryOspf (Issue NMS-8667)
  • Exception in LatencyStoringServiceMonitorAdaptor: IncorrectResultSizeDataAccessException (Issue NMS-8756)
  • Threshold editor help text unclear about comparison operators (Issue NMS-8826)
  • NPE in CdpLink.toString() (Issue NMS-8897)
  • auto-acknoledge random fails (Issue NMS-8901)
  • Near Real-Time Graphing unable to graph Cisco BDI interfaces (Issue NMS-8908)
  • WS-Man throws event 4776 and 4625 with domain user on windows side (Issue NMS-8925)
  • NPE in IpInterfaceScan when processing IPLike rule with IPv6 address (Issue NMS-8934)
  • Statsd: erroneous reporting; aggregators not reset between worker’s runs (Issue NMS-8944)
  • WS_Man datacollection using WQL fails with ‘unsupported element’ (Issue NMS-8955)
  • Enlinkd fails to discovery Bridge Topology (Issue NMS-8973)
  • Typo in linkednode.jsp in panel title (Issue NMS-8981)
  • NRT Graphing does not show SNMP counter values correctly (Issue NMS-9005)
  • Copyright bump 2017 (Issue NMS-9060)
  • opennms.conf can break the snmp-request command (Issue NMS-9068)
  • Jsr160ConnectionFactory doesn’t work with IPv6 address (Issue NMS-9071)
  • WS-Man thresholding reports: No thresholds configured for resource type ‘*’ (Issue NMS-9077)
  • Standalone HTTPS with Jetty doesn’t work with certificate alias (Issue NMS-9084)
  • Correlation engine’s cancelTimer method should be public (Issue NMS-9087)
  • Collection timing enhancements (Issue NMS-9090)
  • Migrate RTC related objects to JAXB instead of Castor (Issue NMS-9100)
  • Pollerd continues to monitor deleted nodes (Issue NMS-9112)
  • Selection of non-disk RRD strategies (Issue NMS-9120)
  • It is possible to perform alarms/notifications actions through the Acks ReST end point without permissions (Issue NMS-9140)
  • SNMP client code can get stuck in infinite retry loops (Issue NMS-9164)
  • incorrect permissions on opennms.service (Issue NMS-9166)
  • Fix the execution of Drools rules when using streaming (CEP, Temporal Reasoning) (Issue NMS-9172)
  • Drools examples fails to load with exception (Issue NMS-9186)
  • SyslogParser should not assume UTC timestamps (Issue NMS-9201)
  • New resource types are not available in the Web UI until it is restarted (Issue NMS-9205)
  • OSPF Router Id not properly updated (Issue NMS-9218)
  • DNS Adapter on Delete Node does not delete DNS record (Issue NMS-9219)
  • Problems associated with SNMP4J affects OpenNMS performance (contention issues) (Issue NMS-9223)
  • Pollerd takes too much time scheduling polling when OpenNMS starts (Issue NMS-9247)
  • Requisition UI breaks when percentage sign is used inside Foreign ID input field (Issue NMS-9269)
  • Make TCP output queue parameters configurable (Issue NMS-9282)
  • Calculation of KSC reports timespans “Last Week” and “This Week” broken (Issue NMS-9325)

Enhancement

  • Upgrade Drools to 6.4.0.Final (from 6.0.1.Final) (Issue NMS-8676)
  • Node detail page is hard to use with a lot of surveillance categories (Issue NMS-8989)
  • SNMP data collection definitions for Cisco ASA5585-SSP-60 devices (Issue NMS-9094)
  • Persist string attributes over the TCP stream (Issue NMS-9126)
  • Add JMX instrumentation for the Drools Correlator to understand the the working memory of each rule-set (a.k.a. engine) (Issue NMS-9145)
  • Upgrade snmp4j to 2.5.5 (Issue NMS-9162)
  • Enhance to MockSnmpAgent to support returning arbitrary SNMP error codes (Issue NMS-9163)
  • Set domain level for DNS adapter (Issue NMS-9222)
  • Allow for choosing ifDescr over ifName for interface directory name (Issue NMS-9240)
  • Improve handling of counter wraps/reset when using Newts (Issue NMS-9252)

by RangerRick at May 22, 2017 09:47 PM

OpenNMS Meridian 2015.1.5 Released

Release 2015.1.5 is the sixth release of OpenNMS Meridian 2015. It contains a few bug fixes and minor enhancements.

The codename for 2015.1.3 is NT.

Bug

  • OpenNMS doesn’t correctly support IPv6 addresses for (jasper) reporting (Issue NMS-5450)
  • Restarting OMNS results in numerous SNMP outage alarms and notifications (Issue NMS-8620)
  • Poller Node Down without outages (Issue NMS-8751)
  • Threshold editor help text unclear about comparison operators (Issue NMS-8826)
  • Invalid graph templates (Issue NMS-8907)
  • Near Real-Time Graphing unable to graph Cisco BDI interfaces (Issue NMS-8908)
  • Copyright bump 2017 (Issue NMS-9060)
  • opennms.conf can break the snmp-request command (Issue NMS-9068)
  • Jsr160ConnectionFactory doesn’t work with IPv6 address (Issue NMS-9071)
  • Migrate RTC related objects to JAXB instead of Castor (Issue NMS-9100)
  • Pollerd continues to monitor deleted nodes (Issue NMS-9112)
  • SNMP client code can get stuck in infinite retry loops (Issue NMS-9164)
  • SyslogParser should not assume UTC timestamps (Issue NMS-9201)

Enhancement

  • SNMP data collection definitions for Cisco ASA5585-SSP-60 devices (Issue NMS-9094)
  • Enhance to MockSnmpAgent to support returning arbitrary SNMP error codes (Issue NMS-9163)

by RangerRick at May 22, 2017 09:46 PM

This Week in OpenNMS: May 22nd, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internal changes, documentation, and web code.

Github Project Updates

  • Internals and Documentation

    • Seth continued working on modernizing our bundled ActiveMQ, Camel, and Karaf versions.
    • Markus continued working on moving requisitions to the database.
    • Ronny updated the documentation for JDK installation, and I added installation documentation for safely dealing with OpenNMS upgrades while OpenNMS is running.
    • ...

May 22, 2017 02:36 PM

OpenNMS Team Wins 5000€ Prize at TM Forum {open}:hack

A group of four students from Southampton Solent University, mentored by Dr. Craig Gallen, used OpenNMS to win the top prize at the TeleManagement Forum {open}:hack competition at the TM Forum Live conference in Nice, France.

{open}:hack Winner Team

Now, a little background is in order. Dr. Gallen founded Entimoss, our OpenNMS partner in the UK and Ireland. He got involved with OpenNMS over a decade ago when he was working on his doctoral thesis entitled "Improving the Practice of Operations Support Sys...

May 22, 2017 11:45 AM

May 18, 2017

OpenNMS Team Wins 5000€ Prize at TM Forum {open}:hack

A group of four students from Southampton Solent University, mentored by Dr. Craig Gallen, used OpenNMS to win the top prize at the TeleManagement Forum {open}:hack competition at the TM Forum Live conference in Nice, France.

Joe Appleton, Jergus Lejko, Michael Sievenpiper and Marcin Wisniewski built the “Port-o-matic” application using OpenNMS to provide a web application for accessing shipping port services, including managing fees and measuring pollution.

Joann O’Brien, the TM Forum’s VP of API’s and Ecosystems, said

They demonstrated fantastic use of all the available technology. They chose a problem relevant to the City of Nice and their home city, that could have a significant impact on society and business by ensuring a better quality environment at ports.

Tarus Balog, CEO of The OpenNMS Group, added:

The hardest thing to get across to people new to OpenNMS is that it is a platform and not strictly an application. The learning curve can be steep and it is hard to see its value straight out of the box. I love the fact that solutions like the “Port-o-matic” demonstrate the power of OpenNMS.

The {open}:hack competition took place over two full days, and in addition to having fun at the conference, the students will be able to enjoy the 5000€ prize money.

by Tarus at May 18, 2017 07:25 PM

OpenNMS Team Wins 5000€ Prize at TM Forum {open}:hack

A group of four students from Southampton Solent University, mentored by Dr. Craig Gallen, used OpenNMS to win the top prize at the TeleManagement Forum {open}:hack competition at the TM Forum Live conference in Nice, France.

{open}:hack Winners

Now, a little background is in order. Dr. Gallen founded Entimoss, our OpenNMS partner in the UK and Ireland. He got involved with OpenNMS over a decade ago when he was working on his doctoral thesis entitled “Improving the Practice of Operations Support Systems in the Telecommunications Industry using Open Source”.

Most of his work was focused on a business solution framework called NGOSS (now Frameworx) developed by the TM Forum for creating next generation OSS/BSS software and systems. Now the TM Forum is the world’s leading trade organization for telecommunications providers and at the time was not very friendly toward open source. He demonstrated how an open source platform like OpenNMS could be used to integrate with and tie together these different interfaces to build a reference implementation for part of the framework. Open source was a new concept for the industry, and we were branded the “open source pirates” at first. But Craig persisted, and in 2011 he was awarded the TM Forum’s Outstanding Contributor Award.

In addition to his persistence and ability to deal with large organizations, Craig is also a great teacher. When the TM Forum introduced its {open}:hack program, he wanted to get involved and he found several interested students at Southampton Solent University.

The goals of {open}:hack are:

  1. Accelerate industry deployment of Forum Open APIs, metamodels and architecture across the industry
  2. Validate existing APIs and provide feedback for future iterations to technical collaboration teams
  3. Create IoT/Smart City & NFV/SDN solutions leveraging the Forum Open APIs
  4. Accelerate the incubation of new digital business opportunities in the areas of 5G Network Services & IoT/Smart City
  5. Create extensions to Forum Open APIs to be shared with industry

Participants were given access to APIs from the TM Forum, Huawei, Salesforce and Vodafone, which included things like data from drones, and tasked with creating something beneficial. Their project was called “Port-o-matic” which created an application for accessing services at shipping ports, as well as measuring environmental factors such as pollution. This was especially relevant to them since Southampton is the UK’s number one cruise port and second largest container port (the Titanic set sail from there).

{open}:hack architecture

Their solution leveraged the power of the OpenNMS platform to tie all of these APIs together and then to provide aggregated data to their web application. It can scale to almost any size using the new OpenNMS “Minion” feature which can distribute data collection and monitoring out to the edges of a network, offloading the need to have all of the functionality in a central location and positioning OpenNMS for the Internet of Things (IoT).

The hardest thing to get across to people new to OpenNMS is that it is a platform and not strictly an application. The learning curve can be steep and it is hard to see its value straight out of the box. I love the fact that solutions like the “Port-o-matic” demonstrate the power of OpenNMS.

It is also interesting to note that the second place prize went to a team from Red Hat. For an organization like the TM Forum that was wary of open source to demonstrate such a change of heart is encouraging, and I credit Dr. Gallen with a lot of that advancement.

{open}:hack Group Photo

So congratulations to Joe Appleton, Jergus Lejko, Michael Sievenpiper and Marcin Wisniewski, the winners of this latest {open}:hack competition, and I look forward to seeing more great things from you in the future.

by Tarus at May 18, 2017 06:48 PM

May 15, 2017

This Week in OpenNMS: May 15th, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internal changes, documentation, and web code.

Github Project Updates

  • Internals and Documentation

    I did a bit more work on packaging issues (OpenNMS and Minion). I also spent a lot of time on flapping smoke tests, and Seth added more test coverage for syslog/kafka/Elasticsearch. Markus did some cleanup on WS-Man support. Marcel and Ronny improved the JMX documentation. Seth continued to work on updating our...

May 15, 2017 04:00 PM

May 12, 2017

Monitoring Websites

Now and then websites are really important for daily business and private life. You can nearly buy everything on websites, find information, meet friends, watch movies and much more. All the worse if your website is not working and you don't notice it or your customer first. But even if you already have some monitoring for webservices or domain name availability, you can't be sure, if your website itself works correct. Maybe the login/logout procedure is broken, some parts of the website are mis...

May 12, 2017 01:50 PM

May 11, 2017

2017 Red Hat Summit

I had never been to a Red Hat Summit before this year. We are exploring running OpenNMS on OpenShift and so Jesse, David and I decided to head to Boston to see what all the fuss was about.

RHSummit - Airline Sign

I noticed a couple of things are different about visiting Boston in spring versus winter. First of all, the weather was quite nice, and second, Boston can be freakin’ expensive.

And Red Hat spared no expense on this conference. This is the premiere event for companies in the Red Hat ecosystem and they obviously wanted to make an impression. I’m an “old guy” and I can remember going to huge shows put on by HP and IBM and this was on par. It took place at the Boston Convention and Exhibition Center (BCEC) which takes up about a half a million square feet. Red Hat used all of it.

RHSummit - Convention Center Sign

Nothing quite demonstrated the size of this conference than the main auditorium. The centerpiece was a huge screen for the presentation flanked by two smaller screens to show the speaker. That was needed since the place was so big you could barely see the person talking.

RHSummit - Main Auditorium Screen

In addition to the general sessions, there were a large number of talks on pretty much anything related to Red Hat products, philosophy and partners. As a major player in “the cloud” there was a lot of emphasis on OpenShift and OpenStack, but the whole range of offerings was covered from Fedora and CentOS to JBoss and Gluster.

As with most tech conferences, there was an expo floor. This one was dominated by the color red.

RHSummit - Expo Floor

I spent a lot of time wandering around talking with people. Over the years a large number of my friends have been hired by Red Hat, and as I’ve curtailed my participation in a lot of the big Linux conferences, it was nice to see them again. I ran into Brian Proffitt and Ruth Suehle near the center of the expo:

RHSummit - Brian Proffitt and Ruth Suehle

It was also nice to run into the Latvian army. The Zabbix crew had a booth and it was cool to see Alexei and Alex again, although it was ironic that I missed them on my trip to Riga (they were actually driving north to Tallinn when I was heading south).

RHSummit - Zabbix Booth

Zabbix, like OpenNMS, is 100% open source and thus not only do we get along, I quite like them and look forward to chatting about the joys and challenges about running an open source business when we meet.

Speaking of meeting, I also got to meet Brian Stinson of the CentOS project.

RHSummit - Brian Stinson from CentOS

We swapped some stories and recounted the strange and funny time when Jerry Taylor, the City Manager of Tuttle, Oklahoma, claimed the CentOS project had hacked his city’s website. Has it been eleven years? Wow.

As part of the conference, Red Hat provided lunch. It was always a pretty hectic time since the show was packed and nothing demonstrated this more than trying to serve lunch to all those people.

RHSummit - Lunch Crowd

As far as conference lunches go, it was above average, but I did find it funny that they only served water to drink (usually there are cans of soda, etc.) I overhead one Red Hat employee say to another, you know, we can afford that gigantic screen but all we get is water?

On Wednesday night, Red Hat purchased a ton of tickets to the Red Sox game at Fenway Park. While I can’t find a reference to actual conference attendance figures, I heard the number 5000 being batted around which was a significant portion of the ballpark (it holds a little over 37,000). They gave us all red baseball caps and you could definitely see them in the crowd.

RHSummit - Fenway Park

For our annual developers conference, Dev-Jam, we have about one-one hundredth the amount of people to see the Twins play, but we also get better seats. (grin)

It was my first time at the historic Fenway Park, and the fans were almost more fun to watch than the game. I also enjoy trying to explain the game of baseball to people from outside the country, and this was made more interesting by some bad blood between the Sox and the Orioles that resulted in the ejection of the Orioles’ pitcher for hitting a batter.

Fenway is relatively close to Cambridge, so I took the opportunity to visit a friend of mine who is a professor. I decided to walk to Harvard Square along the river, where the rowing teams were practicing.

RHSummit - Rowing

Now whenever I see a movie featuring Ivy League students on the water, I’ll know where that was shot.

It was also nice to be able to spend some time with David and Jesse. While I work with David almost daily, we’re so busy that it is hard to find time to talk strategy and plan for the future of OpenNMS. Jesse, our CTO, moved back to Canada after the birth of his son to be closer to family, and it was also nice to have time to spend with him. Walking to dinner one night David took this picture

RHSummit - River and Bridge

which turned out so much better on his iPhone 6S than my Nexus 6P.

I often say that Red Hat, as a company, doesn’t get the credit it deserves since it is headquartered in North Carolina and not Silicon Valley. Our companies share a similar philosophy of taking care of customers, creating great open source software and producing steady growth, versus, say, chasing unicorns. It was wonderful to see that work demonstrated in such a large and professional conference, and I hope next year I’ll get to speak (although I doubt it will be on the big stage).

by Tarus at May 11, 2017 02:49 PM

Easterhack 2017 - Post Mortem

Easterhack 2017

Hello World, We had an Easterhack some days ago, as some of you remember. Marcel Fuhrmann and Markus von Rüden organised the event and Marcel provided a place in his home for the OpenNMS guys to hack, eat, drink, talk and have some fun. Here are a few things what we did (aside from playing Guitar Hero, trying his homemade climbing wall, eating Pizza, and drinking Mead) :)

  • We discussed about best practices for monitoring system processes and the result is published in the opennms.org blog ...

May 11, 2017 01:42 PM

May 09, 2017

Fifteen Years

On Sunday my mother celebrated her 75th birthday.

Although a happy occasion, why is this relevant to an open source blog? Well, it was soon after her 60th birthday in 2002 that I started my first company around OpenNMS.

I did not start OpenNMS, it began in the summer of 1999, with the first code posted on Sourceforge in March of 2000 by a company called Oculan. I started working with Oculan in September of 2001, and in May of 2002 they decided to stop contributing to OpenNMS. I saw the potential, so I asked Steve Giles, the founder and CEO, if I could have the OpenNMS project. He looked at his watch and said if I was off his payroll by Friday, he’d give me the domain names, a couple of servers, and he would sprinkle water on me and I would be the new OpenNMS maintainer.

That was actually the easy part. Explaining to my wife that I had quit my job and started a company “selling free software” was a bit harder.

sortova.com from archive.org circa May 2002

And thus Sortova Consulting Group was born. It was named after my farm. When Andrea and I decided we wanted to have a farm, we first bought raw land. In driving out from Raleigh to work on it we would pass this little farm with a barn, some cows, etc., and on the mailbox was a sign reading “Almosta Farm”. I joked that if that was “almost a farm” then what we had was just “sort of a farm”. Later, when we bought the place where we still live, the name Sortova Farm stuck.

We pronounce it “Sore-toe-va”. Only one customer ever pulled me aside and asked if it really meant “sort of a” consulting group. He laughed when I confirmed that it did.

Considering that I didn’t have any prior business experience, Java experience, or even real Internet access at my home, it is amazing that OpenNMS survived to this day. It is a wonder what you can accomplish with pure stubbornness.

Now my one true superpower is my ability to get the most fantastic people on the planet to work with me. The first group of those came from the OpenNMS community. When I was running Sortova it was the gang that later became the Order of the Green Polo that kept me going, mainly through mailing lists and IRC. In September of 2004 my good friend and business partner David Hustace and I founded the OpenNMS Group, and that corporation is still going strong. In 2009 we mortgaged our houses to buy the copyright to the Oculan OpenNMS code and thus brought all of it back under one organization, and two of the original OpenNMS team at Oculan now work for OpenNMS.

When I visit Silicon Valley I often get to meet some brilliant people, but the joy of this can be offset by the pervasive attitude of focusing on technology simply to make money. I know of a number of personally successful people who built companies, sold them, and then those products vanished into obscurity. Remember VA Linux? Their stock rose over 700% on the first day of trading, but where are they now? Did they ever deliver on their promises to the stockholders?

I want to build with OpenNMS something that will last well beyond my involvement with the project. I’ve gotten it to the point where I am not longer expressly required to make it thrive, but I am still working on its legacy. We want it to be nothing less than the de facto standard for monitoring everything, which is a high bar.

Note that I still would like to make a lot of money, but that isn’t the core driving force of the business. Our mission statement is “Help Customers – Have Fun – Make Money” in that order. If you have happy customers and happy employees, the money will come.

Fifteen years ago I made a leap of faith, in both myself, my family and my friends. I’m extremely happy I did.

by Tarus at May 09, 2017 03:06 PM

Process Monitoring with SNMP

Monitoring processes that don't provide network services is a default use case in network monitoring. Because they aren't providing network services, black box testing won't work- you need an agent on your system providing an inside view of your operating system. The Net-SNMP agent is easy to install and configure on Linux or Unix. It's compatible with any monitoring solution that supports SNMP, such as OpenNMS.

By default, there are basically two methods utilizing Net-SNMP: using the HOST-RES...

May 09, 2017 01:42 PM

May 08, 2017

OpenNMS Horizon 19.1.0 (Octopush) Released

OpenNMS 19.1.0 (code name: Octopush) is now available.

This release contains a large number of bug fixes and infrastructure updates, as well as a few new features:

  • updated Camel, CXF, Drools, Quartz, SNMP4J, and Spring dependencies
  • Drools now persists working memory on OpenNMS shutdown and loads it back up on startup
  • a new syslog parser (RadixTreeSyslogParser) has been introduced which supports most syslog formats and uses Grok for parsing
  • the TCP persister now supports string valu...

May 08, 2017 05:44 PM

May 07, 2017

Privacy and Trash

Meet Sam. Sam is in his early twenties and grew up in Lake Mills, Wisconsin. He graduated from the University of Wisconsin in Madison in 2012. He is currently on vacation in Athens, Greece, with his girlfriend Sara. They managed to find an amazing deal on American Airlines from Minneapolis to Athens for $200 for the both of them, but with taxes and fees that ballooned up to nearly $850.

I have a copy of Sam’s resume, his Gmail address and his phone number. I know how long he’ll be gone and what seats they will be sitting in on their return. In fact, I know a lot more about Sam and Sara (Facebook and its ilk are ubiquitous) but I’m a little uncomfortable revealing as much as I have, so I’ll stop.

It is all because of this:

Sam Boarding Pass

With all the focus recently on the security of devices like those that make up the Internet of Things, what is often forgotten is that traditional paper has huge security issues in today’s connected world.

Airlines still insist on printing first and last names along with record locater codes on boarding passes. That is often all that is required to access a particular reservation. From there you can get information such as e-mail addresses and phone numbers.

This reminds me of when credit cards first came out and to use one the merchant would take an actual imprint of the card on carbon copy paper. Since that included the shopper’s name, complete card number and expiration date, it became easy for thieves to steal this information. At least now almost all receipts include, at most, the last for digits of the card (in case you were wondering, Sam used a Mastercard ending in 3286).

The genesis of this post arose from a more malicious reason. I fly a lot and over the years commercial air travel (which is the only air travel I can afford) has become less of a special occasion and more like taking a bad bus trip. People use the “seat back pocket” as their personal trash can, to the point that I almost never use it myself, even when I get upgraded to first class. Nasty. On this trip, the duration from when the last person got off the inbound plane until we started boarding our flight was less than ten minutes, so trust me when I say little was cleaned between flights.

I don’t blame the airlines. Consumers have spoken, and what they want is cheap airfare, so it is up to us to be respectful of our fellow passengers.

Anyway, when I see folks like Sam leave information like this as trash, I am so tempted to do things like reassign his seat to one in the middle next to the lavatory (it’s an 11 hour flight), or to cancel his flight completely. Lucky for him I believe in karma, and I just can’t bring myself to do it.

The basics of security involve two things: something you have and something you know. We need to apply this to everything that needs to be secure. I get so frustrated with systems in the United States, such as the new “chip” cards being used for credit and debit. Introduced a decade ago in Europe, their systems use “chip and PIN” – something you have, your card, and something you know, your PIN. In the US we are moving to “chip and signature” – something you have, your card, and something anyone can fake in a heartbeat, your signature.

(sigh)

This is especially touchy since two summers ago my spouse had her purse stolen. We immediately canceled and closed all of the accounts, but they were still able to get over $2000 out of our checking account. They used a paper check from another theft and then they cashed it at the bank using her ID. The bank forgot the “something you know” part of security even though they were quite aware that our account had been compromised and the account number changed. Only after the fact did they offer to “flag” transactions on our account for extra scrutiny, and now neither of us carries paper checks, although thieves could probably guess our bank from our ATM debit cards (we did get our money back from the bank).

So be careful. Buy a good shredder. If you need to dispose of paper when traveling, tear it into tiny bits and drop it in the nastiest trash can you can find … and not in the seat back pocket.

by Tarus at May 07, 2017 03:59 PM

May 05, 2017

Restart Detection

Most of today's servers run in virtual environments and Java applications in containers. Sometimes these applications or servers unexpectedly restart and can often go unnoticed because a restart can be much faster then a polling cycle.

If SNMP is enabled, the hrSystemUptime can detect restarts from Windows or Linux machines. Similarly, this concept can also be applied to Java application restart detection via JMX and the JVM uptime metric. This article describes how to use the OpenNMS thresho...

May 05, 2017 12:50 AM

May 03, 2017

LinkedIn

I’m at Red Hat Summit in Boston this week so expect a longer post on the conference later, but I wanted to mention that I’ve reopened a LinkedIn account after an absence of several years. You can find me here:

https://www.linkedin.com/in/tarusbalog

I left the network due to how they were handling privacy issues. I’m still not 100% happy with it now, but I think I can control how much information I share and I do have a need that I think the service can provide.

I was walking in Boston yesterday and I saw a sign for Harvard Medical School. They used to use OpenNMS and I really enjoyed working with the guys who worked there. Most of them have moved on, so I was curious to know where they were and if they were still in the city. It dawned on me that LinkedIn would have helped in this situation.

I don’t like a number of changes that have been made to the site, such as the inability to feature external links (such as to this blog which will remain one of my main ways to communicate) but it may be just my inability to navigate the website. OpenNMS is also on LinkedIn, and it looks like you can “follow” the company as well:

https://www.linkedin.com/company/the-opennms-group

Anyway, let’s give this a go. See you in the toobz.

by Tarus at May 03, 2017 11:51 AM

May 01, 2017

This Week in OpenNMS: May 1st, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internal changes, Minion, and web APIs.

Github Project Updates

  • Internals

    Christian worked more on fixing some issues in resource storage with storeByForeignSource turned on. Ronny did more work on removing old legacy Capsd events. I worked on upgrade-packaging issues when OpenNMS is running. Markus added support for custom JMX URLs, as well as support for using JMX over SSL by default.

  • Minion

    I finish...

May 01, 2017 03:21 PM

April 24, 2017

This Week in OpenNMS: April 24th, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internal changes, documentation, Compass, Minion, and the web UI.

Github Project Updates

  • Internals and Documentation

    Jesse worked on optimizing some of the time series calculation code. I worked on making sure OpenNMS wouldn't upgrade over itself if it is running. Christian added documentation for the Grafana dashboard widget, and Seth documented the new grok-based syslog parser. Alejandro worked on making Dr...

April 24, 2017 03:21 PM

April 20, 2017

New Board, New Claim: Make OpenNMS great again

The new Board

In our last general assembly 13th April 2017 a new board was elected. Dustin Frisch is our new President, Uwe Bergmann our new Vice President and Ronny Trommer our new treasurer.

The assembly itself was very interesting. The discussion was from: “lets dissolve the foundation” to “make it great again”. At the end we decided to keep the foundation and use it further as a plattform to support OpenNMS and its users.

What will happen in the future

The board will have a meeting in a couple of days to define the next steps and the agenda for the next period. Just speaking for me: I am very interested in organizing the next OpenNMS User Conference Europe (OUCE). I spoke to a couple of friends in the last weeks and nearly all of them said they are missing the OUCE.

I think the next OUCE will happen not earlier than next year.

So stay tuned. We will keep you up to date regarding the foundation an the OUCE.

by Uwe Bergmann at April 20, 2017 02:45 PM

April 18, 2017

Reboot & Update Monitor

Security updates are important and need to be installed regularly and as quick as possible.

In Linux distributions like Ubuntu and Debian this is easily done with the apt package manager. Most of the security updates can be installed without rebooting your system. If you upgrade your Kernel it is required to restart your system and Ubuntu and Debian can tell you if a reboot is required. In case you have a fleet of machines to maintain, OpenNMS and Net-SNMP can help you with this task.

This i...

April 18, 2017 01:32 PM

April 17, 2017

This Week in OpenNMS: April 17th, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internal changes, and the web UI.

Github Project Updates

  • Internals

    Alejandro updated our embedded Drools to 6.5.0 and added support for saving/restoring state. Jesse updated the XMP collector to the new collection set API, and cleaned up more legacy collection API usage. Seth added support for customizing the new grok-based syslog parser.

  • Web UI

    Craig did some more work on his asset -> topology adapter...

April 17, 2017 03:14 PM

April 10, 2017

How Version 2.0 Killed Android Wear

I am the happy owner of an LG Urbane smartwatch. Unfortunately, I just upgraded to Android Wear 2.0 and now I can’t use it.

Andrea Wear 2.0 Upgrade

Luckily for me, my smartwatch is not “mission critical”. If I leave it at home by mistake, I don’t turn around to go back to get it. The main thing I use it for is notifications. I like the fact that if it is with me, it will automatically mute my phone and then vibrate when I have a notice. A quick glance at my wrist will tell me if I need to deal with it right this moment, or if it can wait.

The second thing I use it for is to do simple voice searches or to set reminders and timers. Outside of that there are a few apps I use and I like the fact that it tracks my steps, but overall I don’t use a ton of features.

When the notice popped up that I could upgrade, I blindly went ahead and did it. In retrospect, that was stupid, but I often get in trouble rushing out to install the “new shiny”. The upgrade seemed to go fine, and I didn’t think that much about it until lunch.

One of the things I do before heading out to lunch is check the temperature to see if I need a jacket. So I did the usual wrist flick to “wake” the watch and said “Ok Google” to get to the voice prompt.

Nothing happened.

Hrm, I did some research and apparently with 2.0 you have to press the button on the side of the watch to get to the Google prompt. I think this is a huge step backward, because now I have to involve both hands, and I find it ironic that with Android Wear 1.5 I I had to sit through a demo of one-handed gestures over and over again (I often have to re-pair my watch due to reloading software on my phone) and now they’ve thrown “do everything with one hand” out the window.

Anyway, I pressed the button which then brought up the Google Assistant setup screen on my phone. With 2.0 if you want to use voice searches, etc., you must use Google Assistant and you have to give Google access to all of your contacts, calendars etc.

(sigh)

I work hard to “sandbox” my Google activity from the rest of my digital life. It’s not that I think they are evil, it’s just that I don’t want anyone to have that much information on me, well, other than me. I kind of despair for free and open source software solutions in the consumer space. Everyone seems to be rushing to adopt these “always on” digital assistants with absolutely no regard to privacy, and this is causing vendors to lock down their ecosystems more and more. While open source is definitely winning on the server side, I don’t think the outlook has ever been so grim on the consumer side.

There were some upsides with 2.0, such as improvements to the look and feel, but I also found that I didn’t care for the new notification system (I seemed to miss a lot of them – perhaps I needed to change a configuration). But the requirement for Google Assistant was a deal breaker.

I thought about going back to 1.5, which I liked, but I can’t seem to find a factory image. In trying to locate one, I discovered that TWRP does have a version for bass (the codename for the LG Urbane) and I should have installed that and made a backup before upgrading. I contacted LG and they told me it was impossible to downgrade. That’s a load of crap because I could easily sideload the old version if they made it available, but then I’d have to deal with constant upgrade reminders and the few apps I do use would probably stop support for 1.5 to focus on 2.0.

It just isn’t worth it.

I know at least one of my three readers is thinking I should just cave and learn to embrace the Google, but I can’t bring myself to do it. I am eagerly awaiting open source alternatives like Asteriod OS (which just isn’t ready for daily use) and Mycroft (which is supposed to be shipping units this month) but I really don’t think I’ll miss my Urbane enough to spend the time on it.

I plan to sell my Urbane on eBay and I’ve gone back to my previous “dumb” watch (a nice little Frederique Constant I bought on a flight from Dubai to London). It’s kind of a shame since I enjoyed using it, but to be honest I’m not going to miss it all that much.

by Tarus at April 10, 2017 06:18 PM

This Week in OpenNMS: April 10th, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internal changes, and the web UI.

Github Project Updates

  • Internals

    I finished my work on using Java 8's Optional in our JAXB classes and other cleanups. Markus did more work on moving requisitions into the database. Jesse worked on more datacollection infrastructure improvements. Ronny cleaned up a bunch of old events that referred to things like Capsd and Linkd. Antonio worked on fixing some Cisco bridge...

April 10, 2017 02:35 PM

April 05, 2017

Automated Orderings

A self-sufficient printer environment

OpenNMS usually stands for network and server monitoring. But like every good tool you can use (or misuse?!) OpenNMS in other scenarios as well.

If that is a good solution, is completely your choice! :-)

Imagine, you have a big printer environment, distributed in different locations and you don't want to order toner cartridges for every device manually.

So the goal is to let OpenNMS do that job for you!

The configuration for this is quite simple an...

April 05, 2017 11:46 PM

April 03, 2017

This Week in OpenNMS: April 3rd, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internal changes, and the web UI.

Github Project Updates

  • Internals

    I did more work on using Java 8's Optional in our JAXB classes and other cleanups. Seth continued his work on refactoring our syslog parser, and on moving our embedded Quartz scheduler to 2.2. Jesse worked on improving string support in the TCP persister. Dustin worked on improving deletion handling in Provisiond. Antonio fixed more Enlinkd...

April 03, 2017 03:15 PM

March 27, 2017

The Importance of Contributor Agreements

One thing that puzzles me is the resistance within the open source community to contributor agreements. This was brought into focus today when I read that the OpenSSL Project wants to migrate to the Apache 2.0 license from the current project specific OpenSSL license.

In order to do that they need permission from all of the nearly 400 contributors of the project over the last 20+ years, and contacting them will be a huge undertaking. If one person refuses to agree, then they will either have to abandon the effort, or locate that person’s contribution and either remove or replace it.

Many years ago we found out that a company was using OpenNMS in violation of our license. When our lawyer approached them about it, they claimed that they were only using those parts of the code for which we didn’t hold copyright. At that time, early versions of OpenNMS were still copyright Oculan, the company that started the project, and not OpenNMS. Since Oculan wasn’t around anymore it took us awhile to track down the intellectual property, but in the end David and I were able to mortgage our houses to purchase that copyright so that now the project can control all of the code and defend it from license abuse in the future.

But the question arose about what to do moving forward, specifically how should we deal with community contributions? In the past companies like MySQL required all contributors to sign a document with phrases like “You hereby irrevocably assign, transfer, and convey to MySQL all right, title and interest in and to the Contribution” which seemed a little harsh.

I posed this question to the Order of the Green Polo, the de facto project administrators, and DJ Gregor suggested we adopt the Sun Contributor Agreement that we now call the OpenNMS Contributor Agreement, or OCA. This was a straightforward document that asked two things.

First, you attest that you have the right to contribute the code. This is more important than you know, because it helps remove liability from the project should the contribution turn out to be encumbered in some way, such at the author writing it as part of their job and thus it is actually the property of the employer. We allow both individuals and companies to sign the OCA.

Second, you assign copyright to OpenNMS while retaining copyright yourself. This introduces the concept of “dual copyright”. Now some critics will say that this concept hasn’t been tested in court, but there is a long history of authors sharing copyright. Considering that Oracle maintained the agreement in the form of the Oracle Contributor Agreement, it appears that their lawyers were satisfied.

I claim responsibility for the license under which these Contributor Agreements are published: the Creative Commons Attribution-Share Alike License. When DJ suggested the Sun Contributor Agreement I noticed that there wasn’t any license on the agreement itself. I didn’t want to just copy it and change “Sun” to “OpenNMS”, so I contacted Brian Aker who had just moved to Sun with the MySQL acquisition and asked him about it. Soon thereafter the Agreement was updated with the license and we adopted our version of it.

Once we adopted the OCA, I was tasked with tracking down anyone who had ever contributed to OpenNMS outside of the company or Oculan and asking them to sign it. They all did, but I can tell you that I had a hard time tracking down a number of them (people move, e-mails change). I don’t envy OpenSSL at all.

I hope this story illustrates the importance of some sort of Contributor Agreement for open source projects. They don’t have to be evil, and in the end getting your copyright and licensing issues completely sorted out will make managing them in the future so much easier.

by Tarus at March 27, 2017 05:28 PM

This Week in OpenNMS: March 27th, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internals and cleanup, and topology.

Github Project Updates

  • Internals

    I did more work on using Java 8's Optional in our JAXB classes and other cleanups. Seth did more work on refactoring our syslog parser. Markus continued his Provisiond refactor, moving requisition data into the database. Antonio fixed more Enlinkd-related issues.

  • Topology

    Markus and Craig worked more on Craig's asset -> topology m...

March 27, 2017 04:34 PM

March 20, 2017

This Week in OpenNMS: March 20th, 2017

It's time for This Week in OpenNMS! In the last week we worked on various internals, the Minion, and the web UI.

Github Project Updates

  • Internals

    Jesse and I finished converting XML parsing to JAXB. All that's left is to remove some final references to Castor and then it's gone. I also did a bunch of work changing our JAXB objects to use Java 8's Optional class for return values. Antonio did more work on bugs in CDP and OSPF discovery in Enlinkd. Seth did more work on improving...

March 20, 2017 03:40 PM

March 13, 2017

This Week in OpenNMS: March 13th, 2017

Hey there, everybody! I am back from JoCo Cruise 2017 and ready to catch you up on the last 2 weeks of development. In the last 2 weeks we worked on various internals, polling and provisioning, the Minion, and the web UI.

Github Project Updates

  • Internals

    Jesse and I worked more on getting rid of Castor from our codebase, converting XML parsing to JAXB. Alejandro did more work on Drools infrastructure changes to better evaluate and handle memory issues. Seth continued work on impr...

March 13, 2017 03:40 PM

March 09, 2017

Electronic Devices and CPB

With the change in administration in the United States, Customs and Border Protection (CBP) have modified their behavior to include actions with which I don’t agree. These include forcing a US citizen to unlock his mobile device, even though it was a work device and contained sensitive information. I set out to come up with how I will deal with this situation should it arise in the future.

TL;DR My plan is as follows: before I enter the United States, I will generate a long, random password and set that as the encryption password for my laptop and my handy. I will then ssh into an old iMac I have on my desk, store the password in a file, and then shut the computer down. At that point I will not be able to access the information on my device until I return to the office and power on the system.

UPDATE: The EFF has published a detailed guide to help understand your rights at the border.

First off, let me say that until recently I’ve always respected CPB. They have a tough job and everyone I’ve ever met while returning from my travels has been efficient, competent and friendly.

But after the recent “Muslim Ban” fiasco I’ve come to realize that my experience is not universal. I think one of the main problems is this idea that the Constitution stops at the CBP desk, and until you are past it you really aren’t “in America” and thus the Constitution doesn’t apply.

I don’t agree with this interpretation, but it can probably be traced to the actions taken by the US government after 9/11 and the creation of the prison at Guantanamo Bay.

Prior to that, when “bad hombres” were captured by the US government, they fell into one of two categories: criminals or prisoners of war. How each class was treated was fairly well defined. Criminals were processed according to the rule of law, and the treatment of POW’s was covered under the various Geneva Conventions.

The US government decided that those two classifications were inconvenient, and so they ventured into the murky waters of “enemy combatant” and Guantanamo. Their logic goes that since Guantanamo isn’t in the US, US law doesn’t apply, and since these people aren’t members of a foreign country’s military force with which we are at war, then they aren’t POWs. So, the US gets to make up its own rules about how these people are treated.

This is dangerous for a number of reasons. Since nothing is really codified about the treatment and rights of the detainees at Guantanamo, the rules are arbitrary. Also, this opens the door for other countries such as Russia to do similar things without fear of international repercussions. The US has survived for so long because things like this are not supposed to happen, yet here we are.

This thought now extends to the border. Even though a US citizen is being questioned by another US citizen, in the role of a representative of the US government on US soil, somehow the rules of the Constitution are suspended. It’s arbitrary and I don’t buy it. The Constitution codifies a right to privacy in the Fourth Amendment, and it doesn’t go away when entering the country. And it definitely extends to mobile devices, which in today’s world are probably the most personal item people own.

So how can people like me, with almost no political power, resist this threat to our freedom?

I’ve always done little things, like opting out of millimeter wave scans at airports and getting a pat down instead (I’m not shy). If everyone did this the whole system would collapse, and they would find better ways of dealing with security than the security theater we have now. Seriously, if the Israelis don’t use it, it ain’t worth using.

When I turned to the problem of dealing with CBP, my main thoughts went to two devices that I use when traveling: my handy (mobile “phone”) and my laptop. I figured the easiest thing to do would be to just wipe them before coming into the country, but that presents some logistics problems.

For example, I could make a backup of my handy, copy it to a server at home, and then wipe it. The problem is that I have 64GB of storage on the device and I doubt I could transfer a backup in time over, say, a hotel Wi-Fi connection. One of my coworkers uses an iPhone and they thought about wiping their phone and just restoring it from iCloud when they were in the country, but then CBP could require that he turn over his iCloud password.

On my laptop I use whole disk encryption, but I thought about just rsync’ing my home directory and then deleting it before leaving, then again there is the WiFi issue and I really don’t want to have to deal with copying everything back when I’m home.

Then it dawned on me that if I didn’t know the encryption password, then I couldn’t reveal it. The problem became how to create a secure password that I couldn’t remember yet get it back when I needed it.

While my main desktop computer runs Linux Mint, I keep an old iMac on my desk mainly to run WebEx sessions and for those rare times I am forced to use a piece of software not available for Linux. It’s connected to the network, so I can access it remotely. But, if I can access it, I would be lying if CBP asked me for my password and I said I couldn’t retrieve it. Unlike the US Attorney General, I refuse to perjure myself.

Then it dawned on me that I could shut the iMac down remotely and have no way to turn it back on. Thus I could store a passphrase on it, retrieve it when I was back in the country, but until then I would be unable to unlock my devices.

That became the plan. So, the next time I’m returning from overseas, I’ll generate a new, random password. I’ll set that as the whole disk encryption password on my laptop and the encryption password on my handy (note that this is different from the screen-lock password). This will also tie up all of my social network passwords since I use complex ones and store them on those devices. Well, with the exception of my Google account, but since I use two-factor authentication I should be safe as my handy is the device that generates the codes (and I won’t carry any of the backup codes). As long as both of those devices stay powered on, I’ll be able to use them, but once I power them off they will be useless until I get to the office, power on the iMac, and retrieve the passphrase. Note that in order to do that, I’ll be firmly in the US and anyone who wants me to unlock my devices will need a court order.

Which I would respect, unlike CBP. I think the scariest part of the whole “Muslim Ban” incident was when CBP refused to honor court orders. America is built on three branches of government, and when the Executive branch ignores the orders of the Judicial branch we are all in trouble.

I had a two other problems to address, one of which is done. If I’m in the US but my handy is locked, how would I make calls? I might need to call my ride home, etc. To that end I bought a cheap “feature” phone and I’ll just move the SIM card to it when we land.

ZTE Feature Phone

The second issue is that while I should be on solid legal ground concerning my electronic devices, there is nothing preventing CBP from holding me for a long time. Thus the final step is to find an attorney and execute a G-28 form allowing them to represent me. I’m not sure if I need a civil rights lawyer or an immigration lawyer but I’m looking into it. My goal is to be able to notify my attorney when I am coming back into the country, and then send an SMS to them when I am through immigration. If that doesn’t arrive within two hours of my scheduled arrival, they need to come and get me.

I think the thing that bothers me the most about this whole process is the need for it. I’m not a tinfoil-hat conspiracy guy but the actions of the new government have me worried. As I use open source software almost exclusively I know I’m safer than most when it comes to surveillance, and I also don’t expect to run into any problems being an older, white male. But I’d rather be safe than sorry, and the only thing necessary for the triumph of evil is that good men do nothing.

by Tarus at March 09, 2017 02:04 PM

March 07, 2017

Dev Jam 2017: July 16-21 Concordia University

One of my favorite times of the whole year is the week of Dev-Jam, the annual OpenNMS Developers Conference. This year will mark our twelfth meeting, and it has grown quite a bit since our inaugural one in 2005.

For the first time we will be holding Dev-Jam outside of the United States. About a third of the attendees come from other countries and due to recent changes in US immigration policy we couldn’t have people forced to reveal sensitive things such as social media passwords just to come to Dev-Jam.

So, we are holding it at Concordia University in Montreal, Quebec, Canada. Not only is Montreal an awesome city, Concordia is also the alma mater of Jesse White, one of the key architects of OpenNMS. It should be a wonderful venue for the conference.

We have reserved a block of rooms in the historic Grey Nuns Building. Similar to the dorms we have used in the past, every one will get a single room with a shared bathroom. If you would rather stay in a more conventional hotel, there are a number of excellent choices nearby, although I would strongly recommend you consider the dorm as the majority of people will be staying there and it can be quite nice.

Grey Nuns Motherhouse

There are also a number of other events going on in Montreal that week, including a Metallica concert and a fireworks competition, and we will try to do something as a group (baseball is out since the Expos moved to Washington, DC, in 2004).

Space is limited, so if you are the slightest bit interested please let me know and I can reserve you a spot. More details can be found on the wiki and registration is now open.

Hope to see you there, and yes, there will be poutine.

by Tarus at March 07, 2017 08:27 PM

Dev Jam 2017: July 16-21 Concordia University

We are happy to announce our annual meeting Dev-Jam, the annual OpenNMS Developers' Conference. This year will mark our twelfth meeting.

For the first time we will be holding Dev-Jam outside of the United States. About a third of the attendees come from other countries and due to recent changes in US immigration policy, we couldn’t have people forced to reveal sensitive things such as social media passwords just to come to Dev-Jam.

So, we are holding it at Concordia University in Montr...

March 07, 2017 02:42 PM

February 27, 2017

This Week in OpenNMS: February 27th, 2017

In the last week we worked on various internal cleanups and the web UI.

Github Project Updates

  • Internals

    Jesse did more work on cleaning up data collection. Seth continued to work on updating our embedded Karaf to Karaf 4. Jesse and I worked a bunch on converting old Castor-based internal...

February 27, 2017 03:48 PM

February 22, 2017

OpenNMS Horizon 19.0.1 (Ferret-Legging) Released

OpenNMS 19.0.1 (code name: Ferret-Legging) is now available.

This release contains a number of small bug fixes focused mostly on upgrade issues from Horizon 18.

For details on what has changed, see the complete change log.

Breaking Changes/Upgrading

There is a configuration change over 19.0.0 that may require manual changes. See the upgrade notes for full details.

More Info

For complete info on what has changed since Horizon 18, see the release notes.

February 22, 2017 06:44 PM

February 21, 2017

This Week in OpenNMS: February 21st, 2017

In the last week we worked on internals, Minion, the web UI and ReST, and bug fixes.

Github Project Updates

  • Internals

    Ron worked on improving our SSL handling in a number of the SSL-based monitors and detectors. Jesse worked on converting our remaining Castor-based objects to JAXB. He al...

February 21, 2017 04:18 PM

February 19, 2017

Fourteen Years

I just wanted to take a second to thank my three readers for fourteen years of support.

My first post on this blog happened on this date in 2003, and when I wrote it I had little idea I’d still be doing it almost a decade and a half later.

It does seem weird that I still consider OpenNMS a start-up. We took a much different path than a lot of other companies, focusing on our customers instead of fundraising. With our mission statement of “Help Customers, Have Fun, Make Money” and our business plan of “Spend Less Than You Earn” we’ve not only managed to survive but thrive, and both the company and the project have never been stronger. While we are always looking for good investors, this allows us to pick just the right partner.

I’d like to end this with a quote from Michael Seibel of Ycombinator. Actually, it is almost his entire blog post but it really resonated with me.

I’d like to make the point that success isn’t the same as raising a round of financing. Quite the opposite: raising a round should be a byproduct of success. Using fundraising itself as a benchmark is dangerous for the entire community because it encourages a culture of optimizing for short term showmanship instead of making something people want and creating lasting value.

I believe founders, investors, and the tech press should fundamentally change how they think about fundraising. By deemphasizing investment rounds we would have more opportunity to celebrate companies who develop measurable milestones of value creation, focus on serving a customer with a real need, and generate sustainable businesses with good margins.

Optimizing for funding rounds is just as unproductive as optimizing for headcount, press mentions, conference invites, fancy offices, speaking gigs or top line revenue growth with massively negative unit economics.

by Tarus at February 19, 2017 03:44 PM

February 15, 2017

Ulf: My Favorite Open Source Animal

Over at opensource.com they asked “What’s your favorite open source animal?” Hands down, it’s Ulf.

OpenNMS Kiwi: Ulf

When I was at FOSDEM this year, we were often asked about the origin of having a kiwi as our mascot. Kiwi’s are mainly associated with New Zealand, and OpenNMS is not from New Zealand. But Ulf is.

Every year we have a developer’s conference called “Dev Jam“. Back in 2010, a man named Craig Miskell came from NZ and brought along a plush toy kiwi. He gave it to a group of people who had come from Germany, since he had come the furthest east for the conference and they had come the furthest west. They named him “Ulf”.

There was no conscious decision to make Ulf our mascot, it just happened organically. People in the project started treating him as a “traveling gnome“, setting up a wiki page to track some of the places he’s been, and he even has his own Twitter account.

I lost him once. We had a holiday party a few years ago and Ulf went missing. We thought he had been left in a limo, so I dutifully sought out a replacement. I found one for US$9, but of course shipping from NZ was an additional US$80 more, so I bought two. I later found Ulf hiding in the pocket of a formal overcoat I rarely wear (but had the night of the party) so now we have a random array of individual Ulf’s.

Anyway, Ulf manages to represent OpenNMS often, from stickers to holiday cards and keychains. I love the fact that he just kind of happened, we didn’t make a conscious decision to use him in marketing. If you happen to come across OpenNMS at conferences like FOSDEM, be sure to stop by and say “hi”.

by Tarus at February 15, 2017 05:32 PM

February 14, 2017

2017 Europe: Brussels and FOSDEM

This post is about a week overdue, but for the first time in my life I came down with a vicious case of “con crud”. This is a illness that you can get after attending a conference or convention (no reference since the top hits on Google all reference “furries“). This really knocked me out – mainly sinus congestion so severe that my head hurt so bad I couldn’t really sleep. It just laughed at my attempt to treat it with pseudophedrine, and nothing but time seemed to help. Luckily I feel better now and I’m eager to talk about my great time in Brussels at my first FOSDEM.

The Free Open Source Developers European Meeting is probably the largest free software event in the world. This year an expected 8000 people descended on the Université Libre de Bruxelles, and I believe every one of them walked by our stand. It was insane.

I arrived from Riga Friday night and made it to my hotel. My so-called friends had already abandoned me and headed toward the Grand Place and Cafe Delirium, the de facto pre-conference bar.

Cafe Delirium Crowd

Against all odds I managed to catch up with them in the alley outside the bar. Ronny and Markus had come over from Germany, as did Simon and Anya. Jonathan and Craig had come from the UK, and I finally got to meet the amazing Cyrille, a long time OpenNMS contributor who lives in Brussels. There was beer.

Our Gang at Cafe Delirium

We headed over to the university early on Saturday to set up our booth. While this was my first FOSDEM, I was told by a couple of long time attendees that the conference outgrew the venue years ago, with various suggestions for why: from “tradition” to “it’s free”. In any case, it does create an atmosphere that can only be described as special.

FOSDEM Stand

We had a stand in Building K on the second level. This was in a wide hallway surrounding a large auditorium where a number of sessions were held. From the start we got a lot of traffic to the stand, and unlike many conferences the people that stopped seemed genuinely interested in learning about OpenNMS and weren’t just there to check out the swag.

And we had really good swag. In addition to a number of stickers (including the awesome “Ulf Mate” sticker as a play on the “Club Mate” logo and slogan), we had, new for this show, OpenNMS keychain/bottle openers which were a big hit.

OpenNMS Keychains

I also got interviewed for Hacker Public Radio. I don’t remember much of what I said, but people seemed to be into it (grin).

It is seriously difficult for me to describe the crowds. When I needed the restroom, I had to make my way downstairs and then fight my way through a crowd so packed I think it rivaled that year I went to Mardi Gras in New Orleans.

FOSDEM Crowd

But it just lent to the energy and atmosphere of the place. I know from social media that a number of people I know were there that I just missed (looking at you Brian Proffitt) but I did get to see some old friends and I make a few new ones. One person I was happy to meet for the first time was Carol Chen. She is the community manager for ManageIQ, and I first learned about her when Jeff was invited to do an OpenNMS talk at the ManageIQ Design Summit.

Carol Chen and Ulf

She showed up at the stand on Sunday in search of one of our keychains, but we had run out. I had put one away for me but was happy to give it to her. After all, I can always get more.

One thing that sets FOSDEM apart from other open source conferences is the emphasis on “free” software, and some of the social justice causes that naturally follow. Heck, the University has “free” (as in freedom) in its name. Considering that the US President had signed a “Muslim Ban” the week before the conference, it was cool to see this sign on campus.

Refugees Sign

But not all of the fun happened at ULB. Brussels has some beautiful architecture, and just wandering around you might come across a stunning building like this church.

Brussels Church

Nothing is probably as striking as the Grand Place, or central square of Brussels. It is surrounded by tall buildings, some of which represent Guildhalls of various crafts. My friend Daniel pointed out to me that a lot of the money for those buildings probably came from Antwerp during the height of the Hanseatic League. Since the cities of Tallinn and Riga were key players in the Hansa, it kind of brought this European trip full circle.

That’s not to say there aren’t modern things in Brussels. I’ll post this picture without comment.

Mr. Ego Sign

We ended the conference on Sunday with a small group of us meeting for beers and then dinner. Dinner was held at Restaurant Vincent and it was quite good.

Dinner at Vincent's

At the table is Karen Sandler from the Software Freedom Conservancy, me, Lukas and Daniel Ranc from Paris (Daniel teaches at Télécom SudParis and his son is working on his Ph.D.), Cyrille Bollu, Ronny Trommer and Markus von Rüden from OpenNMS, and Spot Calloway from Red Hat.

My only wish is that we could have sat at a round table, since the long table caused conversation to be split into two. I really wanted Daniel and Spot to chat, as Daniel is working on some cool software for education for doing quizzes and surveys in class, and Spot is focused on higher education at Red Hat. But in any case I really enjoyed the conversation, especially one story that Spot told of his college days that I pretty much can’t top (and I pride myself on being able to hold my own when it comes to storytelling).

It was a nice end to an exciting week.

by Tarus at February 14, 2017 09:48 PM

February 13, 2017

OpenNMS Horizon 19.0.0

The long-awaited OpenNMS Horizon 19 (code name: Ice Hockey) is here!

It is chock full of bug fixes and new features, most notably full support for OpenNMS's polling and data collection APIs on the Minion.

New in Horizon 19

  • OpenNMS Minion: Minion systems now have full support for most management features of OpenNMS:
    • Discovery
    • Provisioning
    • Polling
    • SNMP data collection
    • SNMP link discovery
    • Syslog reception (sent to OpenNMS via JMS or Apache Kafka)
    • SNMPv1/v2/v3 trap reception (s...

February 13, 2017 10:30 PM

This Week in OpenNMS: February 13th, 2017

In the last week we worked on internals, Minion, the web UI, and bug fixes.

Github Project Updates

  • Internals

    Jesse added support for requisition providers through a new URL format, requisition://[plugin]. Seth worked on updating OpenNMS's embedded Karaf to 4.x. I fixed an issue with th...

February 13, 2017 04:18 PM

February 09, 2017

OpenNMS Horizon 18.0.4

OpenNMS 18.0.4 (code name: Kiwa Hirsuta) is now available.

This release contains a number of bug fixes and enhancements, including a fix for a deadlock that can be triggered by using the topology UI, and a rather large performance fix for startup if you are using BSM.

Notable Changes

  • many fixes to graph templates and RRD/JRB storage and querying
  • a fix to a filehandle leak in the JIRA ticketer
  • a number of BSM fixes including fixing long startup times when you have a large number of se...

February 09, 2017 11:00 PM

February 06, 2017

This Week in OpenNMS: February 6th, 2017

In the last week we worked on internals, minion, web UI, and bug fixes.

Github Project Updates

  • Internals

    Markus did some more work on auto-compiling Jasper reports. Markus fixed attribute mapping in the JIRA ticketer. Alejandro fixed an issue with spikes in graphs on node reboot. Jesse...

February 06, 2017 04:18 PM