Planet OpenNMS

November 30, 2020

OpenNMS On the Horizon – Documentation, Elasticsearch, Netflow, Traps, Tests, Topology

In the last week we did more documentation migration, Elasticsearch flow storage and queries, Netflow, traps, tests, and topology maps.

Github Project Updates

Internals, APIs, and Documentation
  • Bonnie continued her work on migrating the admin and installation guides to Antora
  • Chandra worked on a fix to Elasticsearch templates when using prefixes
  • Ronny did more work on updating Minion documentation
  • I worked on documentation for JICMP
  • Christian worked on Netflow v9 updates for flowStartMilliseconds and flowEndMilliseconds
  • Marcel did some work on the Slack notification strategy
  • Chandra fixed a bug that could occur when receiving traps on Minion
  • Dustin did some work on flink test infrastructure
Web, ReST, UI, and Helm
  • Stefan worked on updating the flow Elasticsearch and ReST APIs to support querying raw ToS/QoS
  • Patrick continued his work on fixing up topology map application support
Contributors

Thanks to the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Marcel Fuhrmann
  • Matthew Brooks
  • Patrick Schweizer
  • Ronny Trommer
  • Stefan Wachter

Release Roadmap

November Releases

The next OpenNMS release day is December 1st, 2020.

The current expectation is that there will be updates to Horizon 27.0.1 and Meridians 2018 through 2020.

Next Horizon: 28 (Q1 2021)

The next major Horizon release will be Horizon 28.

We're still early in the development cycle for it, but at a high level it is expected to contained our finished work moving OpenBMP's functionality into OpenNMS, as well as enhancements to handle ToS/QoS in flows.

Next Meridian: 2021 (Q2 2021)

With the recent release of Meridian 2020, plans are still tentative.
However, the current plan is that Meridian 2021 will be based on Horizon 28.

We'll know more once development plans start to firm up.

Disclaimer

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

The statements contained herein may contain certain forward-looking statements relating to The OpenNMS Group that are based on the beliefs of the Group’s management as well as assumptions made by and information currently available to the Group’s management. These forward-looking statements are, by their nature, subject to significant risks and uncertainties.

...We apologize for the excessive disclaimers. Those responsible have been sacked.

Mynd you, møøse bites Kan be pretti nasti...

We apologise again for the fault in the disclaimers. Those responsible for sacking the people who have just been sacked have been sacked.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-12733: No support for TLS v1.3 in MailTransportMonitor
  • NMS-13006: flowStartMilliseconds/flowEndMilliseconds for NetFlow v9
  • NMS-13014: Cortex plugin shows only data for counters in a window > 3 hours
  • NMS-13015: Null pointer exception whe minion receives traps
  • NMS-13023: Add clock skew correction mechanism
  • NMS-13024: Check flow sequence numbers to detect missing packets

by RangerRick at November 30, 2020 03:41 PM

November 23, 2020

OpenNMS On the Horizon – Nephron, Provisiond, OpenBMP, sFlow, Netflow, TLS, CIFS, OpenNMS.js, Grafana, Topology

In the last week we did a bunch of work on nephron and flows, fixed a bunch of bugs including provisioning imports and CIFS, fixed timezone support in the Grafana database reports, and more.

Github Project Updates

Internals, APIs, and Documentation
  • Dustin did more work on handling clock skew and other issues in Nephron
  • Ronny did more work on Docker documentation
  • I updated our embedded Jetty to the latest version
  • Bonnie did more work on provisiond and other admin documentation
  • Chandra continued his work on persisting data in the OpenBMP migration
  • Dino fixed an issue with handling ingress/egress with sFlow
  • Christian worked on an issue with DNS hostname enhancement in flows
  • Stefan added support for enabling TLS1.3 on JDKs that support it
  • I fixed an issue with resource leaks in CIFS protocol handling
  • Chandra fixed a provisioning issue with import scanning when new nodes are added
  • Sean made some Nephron updates for Java 11 support
  • David Schlenk made an update to the SNMP interface poller to handle all known ifOperStatus values
  • Christian added flowStartMilliseconds/flowEndMilliseconds support to netflow 9 handling
Web, ReST, UI, and Helm
  • I worked on moving the documentation for OpenNMS.js to Antora
  • I fixed an issue with timezone handling in the report ReST API
  • I updated the Grafana report UI to handle timezones properly
  • Patrick continued his work on fixing Topology Maps and APM
Contributors

Thanks to the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • David Schlenk
  • Dino Yancey
  • Dustin Frisch
  • Jesse White
  • Patrick Schweizer
  • Ronny Trommer
  • Sean Torres
  • Stefan Wachter

Release Roadmap

November Releases

The next OpenNMS release day is December 1st, 2020.

The current expectation is that there will be updates to Horizon 27.0.1 and Meridians 2018 through 2020.

Next Horizon: 28 (Q1 2021)

The next major Horizon release will be Horizon 28.

We're still early in the development cycle for it, but at a high level it is expected to contained our finished work moving OpenBMP's functionality into OpenNMS, as well as enhancements to handle ToS/QoS in flows.

Next Meridian: 2021 (Q2 2021)

With the recent release of Meridian 2020, plans are still tentative.
However, the current plan is that Meridian 2021 will be based on Horizon 28.

We'll know more once development plans start to firm up.

Disclaimer

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

The statements contained herein may contain certain forward-looking statements relating to The OpenNMS Group that are based on the beliefs of the Group’s management as well as assumptions made by and information currently available to the Group’s management. These forward-looking statements are, by their nature, subject to significant risks and uncertainties.

...We apologize for the excessive disclaimers. Those responsible have been sacked.

Mynd you, møøse bites Kan be pretti nasti...

We apologise again for the fault in the disclaimers. Those responsible for sacking the people who have just been sacked have been sacked.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • JS-47: Create Antora documentation for OpenNMS.js
  • NMS-12930: Timezone inconsistency when generating PDF reports from Grafana dashboards
  • NMS-12938: TSS: Cortex Plugin: Use REST API for reading timeseries
  • NMS-12955: sFlow Ingress / Egress
  • NMS-12974: RRD files for SNMP data are not created until a Service Restart
  • NMS-12980: Handle all possible values of ifOperStatus in the SNMP Interface Poller
  • NMS-12994: Provisioning introduction
  • NMS-12995: Document Configure Discovery process
  • NMS-12996: Document requisition process
  • NMS-13011: JCifs leaks memory after upgrade

by RangerRick at November 23, 2020 09:25 PM

November 16, 2020

OpenNMS On the Horizon – Nephron, Flows, Docs, VMware, Minion, BMP, Grafana, Topology Maps

In the last week we worked on Nephron and flows, documentation, VMware updates, Minion certificate handling, OpenBMP migration, Grafana reports, and the topology map.

Github Project Updates

Internals, APIs, and Documentation
  • Dustin did more work on fixing flow data handling in Nephron
  • Bonnie worked on documentation updates for BMP and Provisiond
  • Christian worked on some fixups to VMware asset migration
  • Ronnie worked on Docker, Minion, Sentinel, and flow documentation changes
  • Dustin added some instrumentation to the flow receiver and processor
  • Stefan backported his work to support Minion server certificates to Horizon 27
  • Chandra did more work on migrating OpenBMP's functionality into the OpenNMS core
  • Jesse worked on some Nephron changes to provide near-real-time estimates
Web, ReST, UI, and Helm
  • I worked on fixing time zone handling in Grafana database reports
  • Patrick did more work on fixing topology application map alarm handling
Contributors

Thanks to the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jesse White
  • Patrick Schweizer
  • Ronny Trommer
  • Stefan Wachter

Release Roadmap

November Releases

The next OpenNMS release day is December 1st, 2020.

The current expectation is that there will be updates to Horizon 27.0.2 and Meridian 2020.

Next Horizon: 28 (Q1 2021)

The next major Horizon release will be Horizon 28.

We're still early in the development cycle for it, but at a high level it is expected to contained our finished work moving OpenBMP's functionality into OpenNMS, as well as enhancements to handle ToS/QoS in flows.

Next Meridian: 2021 (Q2 2021)

With the recent release of Meridian 2020, plans are still tentative.
However, the current plan is that Meridian 2021 will be based on Horizon 28.

We'll know more once development plans start to firm up.

Disclaimer

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

The statements contained herein may contain certain forward-looking statements relating to The OpenNMS Group that are based on the beliefs of the Group’s management as well as assumptions made by and information currently available to the Group’s management. These forward-looking statements are, by their nature, subject to significant risks and uncertainties.

...We apologize for the excessive disclaimers. Those responsible have been sacked.

Mynd you, møøse bites Kan be pretti nasti...

We apologise again for the fault in the disclaimers. Those responsible for sacking the people who have just been sacked have been sacked.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-12928: Text for start page
  • NMS-12985: Upgrade script does not migrate VMware metadata
  • NMS-12998: Unable to enable Jaeger tracing in Sentinel
  • NMS-13000: backport Minion certificate management to Horizon 27
  • NMS-13002: Update typo in BMP docs

by RangerRick at November 16, 2020 07:14 PM

November 09, 2020

OpenNMS On the Horizon – Horizon 27, Documentation, OpenConfig gNMI, CircleCI, Nephron

In the last week we released Horizon 27, did more documentation migration work, finished up the OpenConfig gNMI merge, fixed some Horizon 27 bugs (🙃), made some CircleCI improvements, and worked on Nephron performance.

Github Project Updates

Internals, APIs, and Documentation
  • Bonnie and Ronny did more work on migrating our documentation infrastructure to Antora
  • Chandra wrapped up merging his branch to support gNMI OpenConfig
  • I did some work on trying to speed up the tarball assembly portion of the CircleCI build
  • Christian made some fixes to the recent VMware asset changes
  • Sean and Dustin did some work on stabilizing Nephron under heavy load

Contributors

Thanks to the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Ronny Trommer
  • Sean Torres

November Releases - Meridian 2019, Meridian 2020, Horizon 27

November saw bugfix releases from Meridians 2019 and 2020, but most notably saw the (soft) launch of Horizon 27.

Horizon 27 includes a ton of fixes and improvements, but the largest change is the removal of the Remote Poller in favor of a new Minion-based polling mechanism we're calling Application Perspective Monitoring.

Please note that it also includes some changes to VMware metadata handling that we found issues in after releasing. If you use the VMware integration, please hold off on upgrading until 27.0.1 comes out (as soon as we've confirmed the fixes).

Release Roadmap

November Releases

The next OpenNMS release day is December 1st, 2020.

The current expectation is that there will be updates to Horizon 27.0.2 and Meridian 2020.

Next Horizon: 28 (Q1 2021)

The next major Horizon release will be Horizon 28.

We're still early in the development cycle for it, but at a high level it is expected to contained our finished work moving OpenBMP's functionality into OpenNMS, as well as enhancements to handle ToS/QoS in flows.

Next Meridian: 2021 (Q2 2021)

With the recent release of Meridian 2020, plans are still tentative.
However, the current plan is that Meridian 2021 will be based on Horizon 28.

We'll know more once development plans start to firm up.

Disclaimer

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

The statements contained herein may contain certain forward-looking statements relating to The OpenNMS Group that are based on the beliefs of the Group’s management as well as assumptions made by and information currently available to the Group’s management. These forward-looking statements are, by their nature, subject to significant risks and uncertainties.

...We apologize for the excessive disclaimers. Those responsible have been sacked.

Mynd you, møøse bites Kan be pretti nasti...

We apologise again for the fault in the disclaimers. Those responsible for sacking the people who have just been sacked have been sacked.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • HELM-213: Migrate documentation to Antora
  • HELM-252: Helm Plugin on Grafana.com Out of Date
  • NMS-12915: Add gNMI support for OpenConfig
  • NMS-12959: Event Translator debug logging is incorrect
  • NMS-12968: Display the alarm status correctly in topology map for applications
  • NMS-12971: Identify message broker strategies in web "about" page
  • NMS-12975: Nephron Stability Issues at Scale
  • NMS-12979: Alarm (v1 & v2) ReST Service PUT Can't PUT Multiple Things
  • NMS-12986: VMware datacollection failed

by RangerRick at November 09, 2020 08:22 PM

November 02, 2020

OpenNMS On the Horizon – ToS, SSL, gNMI OpenConfig, Docs, Karaf, Web UI, ReST

In the last week we worked on ToS in flows, SSL certificates, gNMI OpenConfig, documentation, Karaf updates, web UI improvements, and ReST.

Github Project Updates

Internals, APIs, and Documentation
  • Christian and Dustin did more work on adding support for ToS handling in flows
  • Stefan worked on a change to support importing server certs into the Minion trust store
  • Bonnie did more work on Provisiond documentation updates
  • Chandra did some final wrap-up of the branch to add gNMI OpenConfig support
  • David did some work on better error-code handling in the SSLCertMonitor
  • Sean worked on updating our embedded Karaf to 4.2.10
Web, ReST, UI, and Helm
  • Jeff made an update to the "About" page to show configured message broker strategies
  • Christian added support for querying the new ToS flow information in Helm
  • Patrick made some fixes to outage handling in the Application Topology View
  • I made a fix to the alarm ReST (v1 & v2) APIs to handle applying multiple values in one PUT request
Contributors

Thanks to the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • David Schlenk
  • Dino Yancey
  • Dustin Frisch
  • Jeff Gehlbach
  • Patrick Schweizer
  • Sean Torres
  • Stefan Wachter

Release Roadmap

November Releases

The next OpenNMS release day is November 3rd, 2020.

The current plan is to release Horizon 27 as well as Meridians 2019 and 2020.

Next Horizon: 27 (Q4 2020)

The next major Horizon release will be Horizon 27.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware meta-data has been moved from assets to the new node meta-data
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • improved SNMPv3 auth configuration
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
Next Meridian: 2021 (Q2 2021)

With the recent release of Meridian 2020, plans are still tentative.
However, the current expectation is that Meridian 2021 will be based on Horizon 28, which is expected to build on Horizon 27 with additional flow and metadata improvements.

We'll know more once development plans start to firm up.

Disclaimer

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

The statements contained herein may contain certain forward-looking statements relating to The OpenNMS Group that are based on the beliefs of the Group’s management as well as assumptions made by and information currently available to the Group’s management. These forward-looking statements are, by their nature, subject to significant risks and uncertainties.

...We apologize for the excessive disclaimers. Those responsible have been sacked.

Mynd you, møøse bites Kan be pretti nasti...

We apologise again for the fault in the disclaimers. Those responsible for sacking the people who have just been sacked have been sacked.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-8484: Add custom string attributes based on indirect and complex SNMP Indices
  • NMS-12897: Filter outages table in Application Topology View
  • NMS-12948: SSLCertMonitor should include more details about the expir(ing|ed) certificate in reason codes
  • NMS-12958: Update Maximum PostgreSQL to allow PostgreSQL 13
  • NMS-12961: Create Horizon 27 Release Notes
  • NMS-12970: Topology Application Map: Outage Table: Clicking on a service should show the outages of the service
  • NMS-12971: Identify message broker strategies in web "about" page
  • NMS-12979: Alarm (v1 & v2) ReST Service PUT Can't PUT Multiple Things

by RangerRick at November 02, 2020 04:33 PM

October 26, 2020

OpenNMS On the Horizon – OpenBMP, Flow ToS, OpenNMS.js, PostgreSQL 13, Outages

In the last week we worked on OpenBMP refactoring, ToS handling in flows, OpenNMS.js updates, documentation, PostgreSQL 13, filtering outages in the application topology view, and bug fixes.

Github Project Updates

Internals, APIs, and Documentation
  • Chandra continued his work on removing OpenBMP as an external dependency
  • Christian worked on adding support for ToS handling in flows
  • I worked on modernizing a bunch of build and runtime dependencies in OpenNMS.js
  • I worked on moving OpenNMS.js documentation to Antora
  • Bonnie worked on installation, minion, and provisioning documentation
  • I updated our PostgreSQL support to include 13
  • David Schlenk made a fix for the opennms startup script not knowing its PID in some cases
Web, ReST, UI, and Helm
  • Patrick worked on updating the application topology view to support filtering outages
Contributors

Thanks to the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • David Schlenk
  • Dustin Frisch
  • Patrick Schweizer

Release Roadmap

November Releases

The next OpenNMS release day is November 3rd, 2020.

The current plan is to release Horizon 27 as well as Meridians 2019 and 2020.

Next Horizon: 27 (Q4 2020)

The next major Horizon release will be Horizon 27.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware meta-data has been moved from assets to the new node meta-data
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • improved SNMPv3 auth configuration
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
Next Meridian: 2021 (Q2 2021)

With the recent release of Meridian 2020, plans are still tentative.
However, the current expectation is that Meridian 2021 will be based on Horizon 28, which is expected to build on Horizon 27 with additional flow and metadata improvements.

We'll know more once development plans start to firm up.

Disclaimer

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

The statements contained herein may contain certain forward-looking statements relating to The OpenNMS Group that are based on the beliefs of the Group’s management as well as assumptions made by and information currently available to the Group’s management. These forward-looking statements are, by their nature, subject to significant risks and uncertainties.

...We apologize for the excessive disclaimers. Those responsible have been sacked.

Mynd you, møøse bites Kan be pretti nasti...

We apologise again for the fault in the disclaimers. Those responsible for sacking the people who have just been sacked have been sacked.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-5879: Debian init.d script wrong postgres dependency
  • NMS-12917: Update Minion installation documentation
  • NMS-12966: service starts / restarts work but spit out an error if configured to wait for startup

by RangerRick at October 26, 2020 05:47 PM

October 19, 2020

OpenNMS On the Horizon – Kafka, BGP/BMP, Sentinel, Vcenter, Time-Series, Helm, and ElasticSearch

In the last week we worked on some Kafka test infrastructure updates, the BMP integration, Vcenter CIM polling, time-series plugins, Helm, and ElasticSearch QoS querying.

Github Project Updates

Internals, APIs, and Documentation
  • Sean worked on bumping Kafka components to 2.6.0
  • Chandra did more work on reworking our BMP integration to not rely on external OpenBMP
  • Alejandro's system property update for Sentinel got updated and merged
  • Christian worked on a fix for Vcenter CIM polling
  • Patrick worked on some fixes for incomplete data in the Cortex time-series plugin
Web, ReST, UI, and Helm
  • I wrapped up the Helm 6 release
  • Christian worked on making the ElasticSearch ReST API support raw and aggregated QoS queries
Contributors

Thanks to the following contributors for committing changes since last OOH:

  • Alejandro Galue
  • Benjamin Reed
  • Chandra Gorantla
  • Christian Pape
  • Patrick Schweizer
  • Sean Torres

Release Roadmap

November Releases

The next OpenNMS release day is November 3rd, 2020.

The current plan is to release Horizon 27 as well as Meridians 2019 and 2020.

Next Horizon: 27 (Q4 2020)

The next major Horizon release will be Horizon 27.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware meta-data has been moved from assets to the new node meta-data
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • improved SNMPv3 auth configuration
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
Next Meridian: 2021 (Q2 2021)

With the recent release of Meridian 2020, plans are still tentative.
However, the current expectation is that Meridian 2021 will be based on Horizon 28, which is expected to build on Horizon 27 with additional flow and metadata improvements.

We'll know more once development plans start to firm up.

Disclaimer

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

The statements contained herein may contain certain forward-looking statements relating to The OpenNMS Group that are based on the beliefs of the Group’s management as well as assumptions made by and information currently available to the Group’s management. These forward-looking statements are, by their nature, subject to significant risks and uncertainties.

...We apologize for the excessive disclaimers. Those responsible have been sacked.

Mynd you, møøse bites Kan be pretti nasti...

We apologise again for the fault in the disclaimers. Those responsible for sacking the people who have just been sacked have been sacked.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-12919: Unable to poll Vcenter CIM - Calling something in OpenJDK11 that has been removed.

by RangerRick at October 19, 2020 06:40 PM

October 12, 2020

OpenNMS On the Horizon – Docs, CI, Detectors and Pollers, Flows, Meta-Data, Helm, Resource Graphs

Apologies for the skipped OOH, I was on vacation last week.

In the last two weeks we worked on documentation, CI, a number of detectors and pollers, flows, meta-data, Helm, and custom resource graphs.

Github Project Updates

Internals, APIs, and Documentation
  • I worked on updating our CI builds to use Cloudsmith for assets
  • I updated the HTTP detector to work with servers that do not send a response reason phrase
  • Christian did more work adding support for setting meta-data in requisitions
  • Dustin worked on improved support for DNS hostname resolution in flow data
  • Bonnie and Ronnie worked on migrating documentation to Antora
  • Alejandro added support for custom.system.properties in the Sentinel Docker image
  • Jeff worked on being able to handle indirect and complex SNMP indices
  • David Schlenk updated some of our WMI bits to support NTLMv2/SMBv2
  • Christian added a fix to handle invalid flow data from some vendor devices
  • Christian fixed VCenter CIM handling on OpenJDK 11
Web, ReST, UI, and Helm
  • I changed the custom resource list to be sorted
  • I did more work on fixing Helm in Grafana 7
  • Chandra made the custom resource graph handle being unable to "graph all" more gracefully
Contributors

Thanks to the following contributors for committing changes since last OOH:

  • Alejandro Galue
  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • David Schlenk
  • Dustin Frisch
  • Jeff Gehlbach
  • Ronny Trommer

October Releases - Horizon 26.2.2, Meridians 2018.1.23, 2019.1.12, and 2020.1.0

Last week we released Horizon 26.2.2, as well as the latest Meridian 2018, 2019, and 2020.

For a complete rundown of the releases, see the October release announcement.

Release Roadmap

November Releases

The next OpenNMS release day is November 3rd, 2020.

The current plan is to release Horizon 27 as well as Meridians 2019 and 2020.

Next Horizon: 27 (Q4 2020)

The next major Horizon release will be Horizon 27.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware meta-data has been moved from assets to the new node meta-data
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • improved SNMPv3 auth configuration
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
Next Meridian: 2021 (Q2 2021)

With the recent release of Meridian 2020, plans are still tentative.
However, the current expectation is that Meridian 2021 will be based on Horizon 28, which is expected to build on Horizon 27 with additional flow and metadata improvements.

We'll know more once development plans start to firm up.

Disclaimer

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

The statements contained herein may contain certain forward-looking statements relating to The OpenNMS Group that are based on the beliefs of the Group’s management as well as assumptions made by and information currently available to the Group’s management. These forward-looking statements are, by their nature, subject to significant risks and uncertainties.

...We apologize for the excessive disclaimers. Those responsible have been sacked.

Mynd you, møøse bites Kan be pretti nasti...

We apologise again for the fault in the disclaimers. Those responsible for sacking the people who have just been sacked have been sacked.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • HELM-247: fix alarm table in Grafana 7
  • NMS-10351: HTTP Detector does not accept a response without a reason as valid
  • NMS-12692: Support hostnames resolution when using aggregated flows
  • NMS-12788: Use newer protocol versions for remote DCOM WMI
  • NMS-12800: Netflow timestamps incorrectly calculated on interfaces with MPLS
  • NMS-12805: Wildcard certificate rejected after upgrade from OpenNMS version 26.1.1 to 26.1.2
  • NMS-12813: Include XML schema for wsman-datacollection-config.xml in assemblies
  • NMS-12918: Allow setting meta data in a requisition
  • NMS-12922: Create a report that matches Horizon 27.0.0 Jira issues with merged pull requests in GitHub
  • NMS-12929: Improvements to Timescale Plugin
  • NMS-12931: sort custom reports
  • NMS-12932: TSS: TimescalePlugin: Create KAR
  • NMS-12933: Update Copyright notice for 2020
  • NMS-12939: Custom Resource Performance Reports returns Missing Parameter: resourceId

by RangerRick at October 12, 2020 07:28 PM

October Releases: Horizon 26.2.2, Meridians 2018.1.23, 2019.1.12, and 2020.1.0

In October we released a Horizon 26 bug fix release, as well as Meridian maintenance releases on the 2018 and 2019 branches. The biggest news, however, is the release of Meridian 2020.

Meridian 2020

Meridian 2020 is the latest in the Meridian series, based on Horizon 26.1.3 plus a number of bug fixes that have gone into the 26 series since its release.

This marks the end of official support for Meridian 2017, although we will continue to make critical updates to the 2017 branch for Powered By OpenNMS customers.

For a high-level overview of what's changed since Meridian 2019, see the Meridian 2020 landing page. The complete release notes are available on meridian.opennms.com.

Meridians 2018.1.23 and 2019.1.12

Meridian 2018.1.23 contained just a small fix for using the correct (location-specific) SNMP config in the SNMP Interface Poller.

Meridian 2019.1.12 includes a number of bug fixes and a few small enhancements.

Horizon 26.2.2

Horizon 26.2.2 contains a number of bug fixes and enhancements, including documentation improvements, time series updates, and a fix for UEI processing.

by RangerRick at October 12, 2020 07:08 PM

September 28, 2020

OpenNMS On the Horizon – Documentation, Thresholding and Requisition Metadata Support, gNMI OpenConfig, Helm

In the last week we did more documentation improvements, worked on metadata support for thresholding and requisitions, added gNMI support to the OpenConfig integration, made Helm impromevements, and did lots of other bug fixing.

Github Project Updates

Internals, APIs, and Documentation
  • Bonnie worked on improving installation documentation for Minion
  • Chandra did more work on adding metadata support for thresholds
  • Ronny and Bonnie worked on moving our documentation rendering to Antora
  • Christian updated the nodeDeleted event to support more log message fields
  • Chandra worked on adding gNMI support to the OpenConfig integration
  • Dustin made improvements to hostname resolution handling in flow queries
  • Mike fixed a bug in the OIA where alarm type was not being set
  • I worked on trying to fix up CircleCI builds to rely less on hitting (flapping) upstream package archives
  • Ronny worked on a new PRIS release using the new documentation workflow
  • Christian worked on adding support for setting metadata in requisitions
  • Jeff worked on an ancient request to add custom string attributes when collecting on complex SNMP indexes
Web, ReST, UI, and Helm
  • I worked on fixing Helm in Grafana 7
  • Jeff updated the Helm nodeResources() function to allow showing the resource label or name, as well as being able to specify a resource type
  • Patrick fixed an issue in flow ReST queries when no data is available
Contributors

Thanks to the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jeff Gehlbach
  • Mark Mahacek
  • Mike Kelly
  • Patrick Schweizer
  • Ronny Trommer

Release Roadmap

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

October Releases

The next OpenNMS release day is October 6th, 2020.

The current plan is to release Horizon 26.2.2 as well as Meridians 2018, 2019, and 2020.

Next Horizon: 27 (Q4 2020)

The next major Horizon release will be Horizon 27.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware meta-data has been moved from assets to the new node meta-data
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • improved SNMPv3 auth configuration
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
Next Meridian: 2020 (Q4 2020)

Meridian 2020 will be based on Horizon 26.1, and should have roughly the same featureset as Horizon 26.1.3 plus a few bugfixes we've made since then.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • HELM-95: Expose Resource Label attribute to Grafana variables
  • NMS-10554: nodeDeleted event should contain more information
  • NMS-12498: Determine migration process
  • NMS-12794: Add support for meta-data on single-DS threshold definitions
  • NMS-12814: Interfaces incorrectly marked as having flows resulting in no data via Helm
  • NMS-12914: Nullpointer in Time Series Integration Layer when query leads to no data.
  • NMS-12917: Update Minion installation documentation
  • NMS-12921: Application link on start page redirects to start page
  • NMS-12923: Integration API: Alarm.type is unset

by RangerRick at September 28, 2020 05:59 PM

September 21, 2020

OpenNMS On the Horizon – Events, Meta-Data, Time Series, SNMP, Helm, Documentation, Perspective Monitoring

In the last week we fixed a bunch of bugs relating to events, meta-data, time series, SNMP, and Helm, updated documentation, and continued to make small perspective monitoring tweaks in preparation for Horizon 27.

Github Project Updates

Internals, APIs, and Documentation
  • Ronny added a log category for the perspective poller
  • Jesse fixed a bug in event matching when a UEI matched across multiple sources
  • Chandra did more work on adding meta-data support to thresholding
  • Patrick fixed some NPE issues in the time series API
  • Chandra made some more fixes for handling broken agents that return the wrong number of OIDs
  • Patrick created some documentation for writing time series plugins
  • Christian enhanced nodeDeleted events to contain additional parameters
  • Chandra fixed the SNMP poller to handle location when looking up config
  • Patrick fixed some queries that Helm uses to determine whether flow data is available
  • Bonnie reworked the documentation on setting up Minion
  • Bonnie worked on providing documentation updates to handle RHEL7-specific requirements
  • I build ARM64 JICMP/JICMP6/JRRD/JRRD2 binaries for RHEL7, RHEL8, and Debian/Ubuntu
Web, ReST, UI, and Helm
  • Ronny updated the application page to direct link to nodes, interfaces, services, and IPs
  • Jeff worked on a fix to show resource labels in Grafana/Helm variables
Contributors

Thanks to the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • David Schlenk
  • Dino
  • Jeff Gehlbach
  • Jesse White
  • Patrick Schweizer
  • Ronny Trommer

Release Roadmap

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

October Releases

The next OpenNMS release day is October 6th, 2020.

The current plan is to release Horizon 26.2.2 as well as Meridian 2019 and 2020.

Next Horizon: 27 (Q4 2020)

The next major Horizon release will be Horizon 27.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware meta-data has been moved from assets to the new node meta-data
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • improved SNMPv3 auth configuration
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
Next Meridian: 2020 (Q4 2020)

Meridian 2020 will be based on Horizon 26.1, and should have roughly the same featureset as Horizon 26.1.3 plus a few bugfixes we've made since then.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • JICMP-25: Build for aarch64
  • NMS-12755: Eventconf with same UEI but differing masks does not follow first-found-wins rule when some events have alarm-data elements and some do not
  • NMS-12818: ArrayIndexOutOfBoundsException thrown by the SNMP Interface Poller
  • NMS-12891: Install guide RHEL instructions are invalid on RHEL 7
  • NMS-12892: Document the MailTransportMonitor
  • NMS-12902: Snmp Interface Poller is not using location specific SNMP Config
  • NMS-12912: Update link to In Memory TS DB
  • NMS-12913: Allow to navigate to monitored items in application status view
  • OIA-32: Create howto for writing tss plugins

by RangerRick at September 21, 2020 04:05 PM

September 14, 2020

OpenNMS On the Horizon – Test Infrastructure, OIA Plugins, Meta-Data for Thresholding

In the last week we wrapped up a few projects, worked on adding meta-data support to thresholding, and fixed a few bugs.

Github Project Updates

Internals, APIs, and Documentation
  • I finally merged the mega-test-refactor branch DJ started a few Dev-Jams ago. Yay!
  • Jesse worked on wrapping up his KAR integration API plugin Maven archetype.
  • Chandra worked on adding support for meta-data in thresholding.
  • Dustin worked on fixing some Telemetryd bugs.
  • I added some more test infrastructure around user authentication.

Contributors

Thanks to the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Chandra Gorantla
  • DJ Gregor
  • Dustin Frisch
  • Jesse White

Release Roadmap

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

October Releases

The next OpenNMS release day is October 6th, 2020.

The current plan is to release Horizon 26.2.2 as well as Meridian 2019 and 2020.

Next Horizon: 27 (Q4 2020)

The next major Horizon release will be Horizon 27.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware meta-data has been moved from assets to the new node meta-data
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • improved SNMPv3 auth configuration
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
Next Meridian: 2020 (Q4 2020)

Meridian 2020 will be based on Horizon 26.1, and should have roughly the same featureset as Horizon 26.1.3 plus a few bugfixes we've made since then.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-12079: Investigate if the InstallerDB could be replaced with something simpler
  • NMS-12655: Rewrite the remote poller backend to use Minion
  • NMS-12817: make allowing legacy MD5 passwords configurable
  • NMS-12896: Telemetryd: a lot of InstanceAlreadyExistsExceptions when starting OpenNMS with default configuration
  • NMS-12901: Grafana step size is ignored when InfluxDB or Cortex is used
  • NMS-12905: Invocation of init method failed; nested exception is org.opennms.netmgt.filter.api.FilterParseException
  • NMS-12910: Add logging configuration for Perspective Poller
  • OIA-29: Create Maven archetype for .kar plugin

by RangerRick at September 14, 2020 06:38 PM

September 08, 2020

OpenNMS On the Horizon – Perspective Monitoring, OpenNMS Plugin Archetype, Documentation

In the last week we continued to work on wrapping up the new perspective monitoring code (formerly remote poller), as well as other bugfixing and release preparations.

Github Project Updates

Internals, APIs, and Documentation
  • Jesse did some work on a Maven archetype for building an OpenNMS plugin.
  • Christian, Dustin, and Patrick did more work wrapping up the new Minion perspective poller.
  • Bonnie fixed some configuration documentation in the admin guide.
  • Patrick worked on some cleanup in the timeseries integration layer.
  • Chandra made some improvements to handling certain malformed SNMP reponses.
  • Bonnie made a bunch of updates to the provisioning documentation.
  • Patrick made some fixes related to the application topology manager.
Web, ReST, UI, and Helm
  • Chandra fixed a bug triggering custom reports.
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jesse White
  • Patrick Schweizer

September Releases: Horizon 26 and Meridian 2017 through 2019

In September we released updates to Horizon and all supported Meridian releases.

Horizon 26 got a bump to 26.2 thanks to the addition of support for a new telemetry adapter which can collect from OpenConfig over the gRPC Juniper Telemetry Interface.

Meridian 2019.1.11 got a bunch of bug fixes and a few handy enhancements.

Meridians 2017.1.26 and 2018.1.22 each got a critical subset of the bug fixes that went into 2019.1.11.

Release Roadmap

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

October Releases

The next OpenNMS release day is October 6th, 2020.

The current plan is to release Horizon 27.0.0 as well as Meridian 2019 and 2020.

Next Horizon: 27 (Q4 2020)

The next major Horizon release will be Horizon 27.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware meta-data has been moved from assets to the new node meta-data
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • improved SNMPv3 auth configuration
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
Next Meridian: 2020 (Q4 2020)

Meridian 2020 will be based on Horizon 26.1, and should have roughly the same featureset as Horizon 26.1.3 plus a few bugfixes we've made since then.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-12773: Anomaly Detection Experiment
  • NMS-12774: Anomaly Detection - Get the consumer working
  • NMS-12823: Application Topology Provider Status
  • NMS-12860: Remote Poller: Documentation
  • NMS-12862: Incorrect TSS settings
  • NMS-12865: Remote Poller: Clearify perspecive labeling
  • NMS-12874: Remote Poller: Rename to Perspective Poller
  • NMS-12881: Remote Poller: Clear outages after removing from application
  • NMS-12882: Perspective Poller shows in tracing as RemotePollerNG
  • NMS-12886: Application Perspective Monitoring: create Application status page
  • NMS-12887: Application Perspective Monitoring: NPE when starting OpenNMS
  • NMS-12889: Application Perspective Monitoring: OpenNMS refuses to start if service is referenced by two applications
  • NMS-12893: Additions to Application Perspective Monitoring docs

by RangerRick at September 08, 2020 07:39 PM

September 03, 2020

Coming October 2020: Meridian 2020!

Based on Horizon 26.1.3, here’s a preview of some of the features we’re excited to bring to Meridian 2020:

Enriched Flows, Better Messaging with Kafka
Configure flows to include node metadata before forwarding to Kafka so that you can aggregate flow data or integrate it with your own custom database to persist all flows.

New Kafka event and alarm publishing and the ability to configure communications using a single topic rather than one per module mean better and faster Kafka performance.

BGP Monitoring Protocol (BMP) Functionality
Support for the BGP Monitoring Protocol (BMP) in the streaming telemetry feature (telemetryd) allows for advanced monitoring and management, and provides a convenient way to monitor BGP routing information on the routing device.

Streamlined Web UI
Meridian 2020 marks the beginning of our long-term project to update the Web UI, starting with the navbar notification alert, a reorganized user menu, and new features in the requisitions UI.

Better Minion Operation in Containerized Environments
Easier configuration for Minions running in container environments like Docker, Kubernetes, or CRI-O, through a single entry point. Docker-specific Minion updates include 50% smaller Minion container sizes, native ICMP support for improved performance, and ARM support.

To learn more about Meridian 2020, contact sales@opennms.com.

by Bonnie at September 03, 2020 06:22 PM

September 02, 2020

September Updates for Meridian 2017 through 2019 and Horizon 26

OpenNMS is released on a monthly schedule, the first Tuesday of the month. The next scheduled release will be October 6th.

In September, we released updates for all supported OpenNMS releases.

Horizon 26.2.1

Release 26.2.1 is the eighth release in the Horizon 26 series.

(Note: 26.2.0 was released in the September release window, but was missing some important fixes because of a few missed branch merges. 26.2.1 contains all of the fixes that were targeted to 26.2.0.)

It contains a bunch of bug fixes and improvements, including adding support for a new telemetry adapter supporting collection from OpenConfig over the gRPC Juniper Telemetry Interface.

The codename for 26.2.0 was Kabuki.
The codename for 26.2.1 is Maskne.

Bug Fixes
  • Slack-compatible notification strategies expect "url" for switch name, should be "-url" (Issue NMS-10552)
  • Resource Graph properties throws exception if label starts with a number (Issue NMS-12793)
  • importing a provisioning group erases path outages (Issue NMS-3608)
  • Can't install Horizon on Ubuntu 20.04 LTS (Issue NMS-12693)
  • opennms.pid missing when started by Systemd (Issue NMS-12769)
  • Wildcard certificate rejected after upgrade from OpenNMS version 26.1.1 to 26.1.2 (Issue NMS-12805)
  • AbstractXmlCollectionHandler.parseString() doesn't handle json content (Issue NMS-12812)
  • Syslogd is sending new suspect events with null IP Address (Issue NMS-12824)
  • NPE while running AlarmLifecycleListenerManager (Issue NMS-12825)
  • Fix CollectdTest mock'ing errors (Issue NMS-12828)
  • Fix JMX datacollection config generator test (Issue NMS-12829)
  • Remove unused import in BsonUtils (Issue NMS-12830)
  • Update mockito/powermock dependencies (Issue NMS-12831)
  • Response Time Summary database report missing latency caluculation (Issue NMS-12837)
  • SslContextFactory needs to be changed to SslContextFactory.Server in jetty.xml (Issue NMS-12847)
  • Wrong startup command for Minion running with Docker and health check issues (Issue NMS-12872)
Enhancements
  • Create OpenConfig Client to collect/digest Telemetry data (Issue NMS-12729)
  • Backport OpenConfig to release-26.x (Issue NMS-12867)
  • Support encryption for SNMP V3 credentials (Issue NMS-12753)
  • snmpv3 need support for SHA256 and SHA512 Authentication Protocol (Issue NMS-12782)
  • Add documentation for OpenConfig feature (Issue NMS-12857)

Meridian 2019.1.11

Release 2019.1.11 contains a bunch of bug fixes and a few handy enhancements.

The codename for 2019.1.11 is Haumea.

Bug Fixes
  • Slack-compatible notification strategies expect "url" for switch name, should be "-url" (Issue NMS-10552)
  • Can't install Horizon on Ubuntu 20.04 LTS (Issue NMS-12693)
  • opennms.pid missing when started by Systemd (Issue NMS-12769)
  • Resource Graph properties throws exception if label starts with a number (Issue NMS-12793)
  • Wildcard certificate rejected after upgrade from OpenNMS version 26.1.1 to 26.1.2 (Issue NMS-12805)
  • AbstractXmlCollectionHandler.parseString() doesn't handle json content (Issue NMS-12812)
  • Syslogd is sending new suspect events with null IP Address (Issue NMS-12824)
  • NPE while running AlarmLifecycleListenerManager (Issue NMS-12825)
  • Fix CollectdTest mock'ing errors (Issue NMS-12828)
  • Fix JMX datacollection config generator test (Issue NMS-12829)
  • Remove unused import in BsonUtils (Issue NMS-12830)
  • Response Time Summary database report missing latency caluculation (Issue NMS-12837)
  • SslContextFactory needs to be changed to SslContextFactory.Server in jetty.xml (Issue NMS-12847)
  • Custom Resource Performance Reports is broken (Issue NMS-12870)
Enhancements
  • Support encryption for SNMP V3 credentials (Issue NMS-12753)
  • Syslog should fallback on source address if hostname is not DNS resolvable. (Issue NMS-12846)

Meridians 2017.1.26 and 2018.1.22

Both the Meridian 2017 and 2018 releases have the same changes applied.

The codename for Meridian 2017.1.26 is Philadelphia meridian and the codename for Meridian 2018.1.22 is Firenado.

Bug Fixes
  • Can't install Horizon on Ubuntu 20.04 LTS (Issue NMS-12693)

  • Wildcard certificate rejected after upgrade from OpenNMS version 26.1.1 to 26.1.2 (Issue NMS-12805)

  • SslContextFactory needs to be changed to SslContextFactory.Server in jetty.xml (Issue NMS-12847)

  • Slack-compatible notification strategies expect "url" for switch name, should be "-url" (Issue NMS-10552)

by RangerRick at September 02, 2020 09:46 PM

August 31, 2020

OpenNMS On the Horizon – Slack Notifications, SNMPv3 Encryption, OpenConfig, Minion, Application Topology

In the last week we merged some old pending enhancements and cleanups, worked more on wrapping up the OpenConfig and Minion-based remote poller features, and fixed a bunch of bugs.

Github Project Updates

Internals, APIs, and Documentation
  • I wrapped up merging Jeff & Markus's improvements to the Slack/Mattermost notification strategy
  • I did more work on the changes to enforce SHA-256 passwords
  • Stefan added support for additional java config options on the Minion
  • Chandra and Jesse did some work on OpenConfig support
  • Christian and Dustin continued to work on polishing up the new remote poller Minion code
  • Bonnie worked on cleaning up the provisioning documentation
  • Chandra worked on adding more hash algorithms to SNMPv3 support
  • Ronny fixed some issues with Minion health check in Docker
  • Chandra fixed a bug in feature dependencies from the gRPC rework
Web, ReST, UI, and Helm
  • Patrick worked on some improvements to service status propagation in the Application Topology Provider
  • Chandra fixed a bug in running custom reports
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jeff Gehlbach
  • Jesse White
  • Markus von Rüden
  • Patrick Schweizer
  • Ronny Trommer
  • Stefan Wachter
  • Zoë Knox

Release Roadmap

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

September Releases

The next OpenNMS release day is September 1st, 2020.

Currently we expect new bugfix releases from Meridians 2017 through 2019.
Also we'll be releasing Horizon 26.2, which will add OpenConfig support on top of the existing Horizon 26 stable codebase.

Next Horizon: 27 (Q4 2020)

The next major Horizon release will be Horizon 27, due in 4th quarter 2020.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware meta-data has been moved from assets to the new node meta-data
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • improved SNMPv3 auth configuration
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
Next Meridian: 2020 (Q4 2020)

Meridian 2020, based on Horizon 26.1, has been pushed back a month for some more polish.

We now hope to release it for the October release window.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-10552: Slack-compatible notification strategies expect "url" for switch name, should be "-url"
  • NMS-12780: RemotePoller: remove unused response time from Liquibase-Changeset
  • NMS-12782: snmpv3 need support for SHA256 and SHA512 Authentication Protocol
  • NMS-12837: Response Time Summary database report missing latency caluculation
  • NMS-12843: Remote Poller: add remote polling details to service page
  • NMS-12847: SslContextFactory needs to be changed to SslContextFactory.Server in jetty.xml
  • NMS-12864: Remote Poller: Cleanup
  • NMS-12866: Remote Poller: Details pages show remote outages as native
  • NMS-12867: Backport OpenConfig to release-26.x
  • NMS-12868: Remote Poller: Add graph definitions for response times
  • NMS-12869: Remote Poller: Add distributed tracing
  • NMS-12870: Custom Resource Performance Reports is broken
  • NMS-12871: Remote Poller: Fix event definition for remote poller nodeLostService events
  • NMS-12872: Wrong startup command for Minion running with Docker and health check issues
  • NMS-12877: Remote Poller: NPE while attempting to start RemotePollerNG
  • NMS-12878: Remote Poller: Make backend more resilient
  • NMS-12880: Unable to install feature 'dominion-grpc-client'

by RangerRick at August 31, 2020 04:26 PM

August 24, 2020

OpenNMS On the Horizon – Build Updates, Remote Poller Minion, OpenConfig

In the last week we did more bugfixes, merged a bunch of build updates, continued work on moving the remote poller to the Minion, and integrated OpenConfig support.

Github Project Updates

Internals, APIs, and Documentation
  • I merged a bunch of Ron Roskens' cleanup in tests and dependencies as part of his work to get OpenNMS building under JDK 11
  • I finally finished my branch fixing up Systemd PID-handling and startup
  • Patrick, Dustin, and Christian did more work on trying to wrap up transitioning the remote poller to the Minion
  • I prepped branches and merging for a Meridian 2020 release
  • Jesse did some work on making a Maven archetype for creating OIA plugins
  • I did a few more cleanups and improvements to CircleCI workflows and smoke tests
  • Jesse merged the OpenConfig telemetryd branch
Web, ReST, UI, and Helm
  • Bonnie updated the requirements to Helm in preparation for a 5.0.3 bugfix release
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jesse White
  • Patrick Schweizer
  • Ronald Roskens

Release Roadmap

Note that this is just based on current plans; dates, features, and releases can change or slip depending on how development goes.

September Releases

The next OpenNMS release day is September 1st, 2020.

Currently we expect new bugfix releases from Meridians 2017 through 2019.
Also we'll be releasing Horizon 26.2, which will add OpenConfig support on top of the existing Horizon 26 stable codebase.

Barring complications, we also expect to release Meridian 2020 (based on Horizon 26) during the September release window.

Next Horizon: 27 (Q4 2020)

The next major Horizon release will be Horizon 27, due in 4th quarter 2020.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware meta-data has been moved from assets to the new node meta-data
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • HELM-251: update requirements
  • NMS-10627: Errors in runjava script
  • NMS-12676: Dynamic service scheduling: Service lifetime
  • NMS-12729: Create OpenConfig Client to collect/digest Telemetry data
  • NMS-12769: opennms.pid missing when started by Systemd
  • NMS-12828: Fix CollectdTest mock'ing errors
  • NMS-12829: Fix JMX datacollection config generator test
  • NMS-12830: Remove unused import in BsonUtils
  • NMS-12831: Update mockito/powermock dependencies
  • NMS-12838: Add events for new Application Model
  • NMS-12842: Remote Poller: add remote outages to UI
  • NMS-12849: Remote Poller: remove not longer used role ROLE_REMOTING
  • NMS-12850: Remote Poller: remove references to old remote poller from documentation
  • NMS-12852: Remote Poller: remove old Distributed Status Summary UI
  • NMS-12854: Prepare Meridian 2020 release
  • NMS-12855: create Meridian 2020 branches
  • NMS-12856: create PoweredBy 2020 repo
  • NMS-12857: Add documentation for OpenConfig feature
  • NMS-12858: Create Meridian 2020.1.0 Documentation from Horizon 26 "What's New"
  • NMS-12859: Remote Poller: Remove LocationSpecificStatus
  • NMS-12861: Remote Poller: Fix the remote flag

by RangerRick at August 24, 2020 03:34 PM

August 20, 2020

Windstream Case Study: When Automatic Provisioning Is Not Feasible

OpenNMS' automatic provisioning allows for rapid discovery and persistence of tremendous numbers of nodes, interfaces, and services. But what happens when device design prevents a customer from using automatic provisioning, and manual discovery would be tedious and time-consuming?

You leverage OpenNMS' customizability and metadata DSL feature to discover those devices.

The Customer: Windstream

Windstream provides high-speed broadband Internet, phone service, and digital TV packages to residential customers, and products and services for businesses and government agencies.

The Problem: Device Design Prevents Auto-Discovery

Windstream deploys Calix C7 Multiservice Access Systems (C7s) in its network. C7s work in clusters, where all members of the cluster share the same IP address. Data collection from C7s is complicated, as each device has multiple internal SNMP agents, each of which has its own credentials and settings, all exposed on the same IP address and different ports.

During automatic provisioning, OpenNMS expects each device to have its own IP address and SNMP agent. With C7s, there is no way to build inventory in OpenNMS’s traditional, automated way.

Making SNMP requests against C7 devices is expensive: an overwhelming number of requests, many of which would be discarded if the devices are busy, also means an unacceptable impact on performance.

Network abstract visualization

OpenNMS Solution: Customize Using Meta-Data Feature to Manually Populate Inventory

The OpenNMS Group used its Meta-Data DSL (domain specific language) feature to populate the inventory using the node’s endpoint of the ReST API. A custom script uses metadata syntax to explicitly identify the ports needed on the physical interface during the provisioning process.

At runtime, the script overwrites the actual port numbers, allowing collection from each SNMP agent, even when other agents on the device share the same IP address. Instead of retrieving data from an entire SNMP table, the script adds the specific SNMP or physical interfaces directly to the database without querying the device, so the Collector can retrieve performance metrics through SNMP, only from those interfaces.

Read the Windstream Case Study.

by Bonnie at August 20, 2020 06:12 PM

August 17, 2020

OpenNMS On the Horizon – Bug Fixing, Remote Poller in Minion, OpenConfig, Meridian 2020, Dev-Jam Videos Live

In the last week we did more bug fixing, continued the work of moving remote poller funcitonality to the Minion, OpenConfig support, Meridian 2020 prep, and more.

Also, the presentations for Dev-Jam 2020 are now live.

Github Project Updates

Internals, APIs, and Documentation
  • I continued to work on fixing up some guts of the OpenNMS systemd startup
  • Patrick, Dustin, and Christian did some more work on moving the remote poller to the Minion
  • Dustin updated our scripts to all use #!/usr/bin/env bash rather than #!/bin/bash
  • Jesse and Chandra worked on OpenConfig support, including creating a new telemetry API for configuring connections to external tools
  • Chandra worked on a bug where Syslogd would send new suspect events with a null IP
  • Stefan added support for passing arbitrary java options to the JVM startup on Minion using confd
  • Patrick fixed an issue with handling resource graph properties
  • I worked on prepping for Meridian 2020, based on Horizon 26
  • Patrick worked on an issue with propagation of application topology provider statuses
Web, ReST, UI, and Helm
  • Jeff worked on fixing the example jetty.xml to match the current version
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jeff Gehlbach
  • Jesse White
  • Patrick Schweizer
  • Stefan Wachter

Dev-Jam 2020 Presentations Now Live

Dev-Jam is a time for developers of OpenNMS to get together, socialize, and work on something cool.
This year (of course) Dev-Jam was virtual, but we continued the tradition of Friday presentations to show what was worked on during the week.

The presentation videos are now live on our YouTube channel.
There were 11 presentations, ranging from integrations to notifications to testing and UX.
Check them out here!

Calendar of Events

Upcoming Releases - September 1st, 2020

The next OpenNMS release day is September 1st, 2020.
Currently we expect new bugfix releases from Meridian 2019 and Horizon 26.

Also coming up next is Horizon 27.
While we can't yet say when it will be released, it's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware metadata has been moved from assets to the new node metadata
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
  • BMP improvements

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-12693: Can't install Horizon on Ubuntu 20.04 LTS
  • NMS-12792: Remote Poller: Add outages for remote poller status changes
  • NMS-12793: Resource Graph properties throws exception if label starts with a number
  • NMS-12805: Wildcard certificate rejected after upgrade from OpenNMS version 26.1.1 to 26.1.2
  • NMS-12812: AbstractXmlCollectionHandler.parseString() doesn't handle json content
  • NMS-12821: Remote Poller: Change monitoring locations UI to reflect model changes
  • NMS-12822: Remote Poller: Change Admin UI to reflect Application model changes
  • NMS-12824: Syslogd is sending new suspect events with null IP Address
  • NMS-12840: Hardcoded path to bash
  • NMS-12844: Remove polling package selection from application
  • NMS-12851: Remote Poller: remove scanreports UI and ReST endpoint

by RangerRick at August 17, 2020 03:34 PM

August 13, 2020

Dev Jam 2020 – It’s a Wrap!

OpenNMS’ week-long annual IT conference, hackathon, and get-together for the global OpenNMS development community wrapped up Friday, July 31.

Event Held in Minecraft

The Covid-19 pandemic forced the normally in-person Dev Jam online, but with a twist. The event was held in Minecraft, customized with OpenNMS in mind.

“OpenNMS has a community full of amazing people who make DevJam feel more like a family reunion than a conference. We really enjoy getting together to work on pet projects and catch up,” says Jessica Hustace, Marketing Director at The OpenNMS Group.

“Since we decided to carry on the tradition of DevJam virtually, we wanted to do our best to recreate the feeling of being together in person. We needed a platform-agnostic immersive environment that we could customize and that was relatively accessible for everyone. Minecraft met all three criteria. It was really fun to see everyone customize their avatar and run around the world together!”

“Minecraft has a lot of advantages for something like this. It's easy to learn and fun, and the possibilities are endless in its open world,” says Zoë Knox, VP of Engineering at OpenNMS. Several OpenNMS staff had experience running Minecraft servers and writing plugins, making it a good choice for customization.

“We wanted it to look "grand" so we chose a stunning map called Monument Valley – available online – as the starting point,” she adds. “A small group of us then hollowed out some of the buildings and built interior conference spaces, an outdoor stage and benches, and other interesting elements. I created a big "DevJam 2020" sign floating in front of the main building. It wasn't particularly hard to do but it did take most of two weeks of spare time to get everything how we wanted.”

shows the minecraft environment for opennms devjam 2020

The group added a Mumble server and MumbleLink mod to get positional audio to create a very intuitive experience.

“You walk towards someone, and they get louder. Walk away, and you don't hear them anymore,” Zoë says. “This gave us the multiple conversations that we wanted, and the natural feel of being in the room with people.”

Chris Manigan, Director of Information Technology at OpenNMS, built the Minecraft server in Amazon Web Services (AWS). He did not allocate a lot of resources at first, as he didn’t know how popular it would be.

“After our first real session, I quickly realized the memory I had allocated for Minecraft was not enough, so after that session I doubled the memory Minecraft was allowed to use,” he says. “It was barely enough, but it got us through the week and even allowed us to host a couple of plugins that were developed as projects for Dev Jam. I believe we had more than 30 people connected at one point, all lighting off fireworks and throwing tridents at each other.”

Chris says that the Minecraft server remains online to demo the plugin projects that were developed during the week. “I would like to see the server live on for a while, and possibly be revisited for future Dev Jams,” he adds. “I would build it out a little better for next time. Perhaps the first OpenNMS>Minecraft plugins developed will inspire more in the future.”

Virtual Dev-Jam 2020 Tarus POV

Exciting New Projects

“I look forward to DevJam every year because I enjoy seeing so many other passionate people like me about building open-source software,” says CTO Jesse White. “While we weren’t able to meet face to face this year, we were still able to come together as a community, share some virtual experiences, and expand the platform in exciting ways.”

Dev Jam always ends with presentations of some of the work people accomplished during the week. This year, we live-streamed twelve presentations on our Twitch channel, with over 100 live views. The projects covered a wide range of topics:

In keeping with the Minecraft theme, two presentations made use of the game.

Zoë created a network operations centre in Minecraft, that displayed real-time data from an OpenNMS instance.

A demonstration Jesse’s project to integrate pager duty saw him receive an alert on his phone when more than 10 players entered a certain zone in the Minecraft world.

Other DevJam projects included:

  • Porting DJ’s integration test enhancements to the head of the tree (Ben)
  • Strides in getting the project to build with JDK11+ (Ron)
  • Enhancing the SNMP Interface Poller to handle failed SIP dial peers better (David)
  • Interfacing with OpenNMS’ REST API from Python (Mark)
  • Improved support for collecting metrics from Prometheus’ node_exporter (Marcel)
  • Migrating node asset fields to the new meta-data tables (Christian)

“That’s quite a lot of activity for a single week!,” says Jess. “I am thrilled to be a part of the team at The OpenNMS Group and look forward to continuing to work with our open-source community and customers that help support the project.”

To read about more of the presentations worked on at Dev Jam, check out the board on Discourse.

A Huge Success

“Overall I think Virtual DevJam was a big success!,” says Jess. “Everyone had fun choosing skins for their character, throwing tridents and shooting bows at each other, and building a minecart rollercoaster at the infamous Mouse Bar. Despite not being physically in the same room, it certainly felt like we were all together. I take that as a win.”

Until Next Year

Dev Jam has evolved over the years, and will continue to do so, particularly as OpenNMS grows.

Keeping us informed for next year, Jess states, "Hopefully we can all join together next year for a more traditional DevJam! Until then, I hope everyone can stay safe and well. DevJam is an important part of the OpenNMS family. We love hacking on the project together and building our community into one of the most loving and welcoming spaces on web. Finally, I'd really like to thank everyone in the team who pitched in to make the first virtual DevJam successful."

by Bonnie at August 13, 2020 05:46 PM

August 10, 2020

OpenNMS On the Horizon – Bug Fixes, Documentation Updates, Remote Poller, Virtual Dev-Jam 2020

Sorry for the 2-week cycle on this one, but I was on vacation last week. Also -- DEV-JAM happened! So there's a lot to catch up on.

In the last 2 weeks we did more bug fixing work, continued to chip away at moving the remote poller to Minion, and then -- of course -- Dev-Jam projects.

Github Project Updates

Internals, APIs, and Documentation
  • Jeff updated the admin guide's Elasticsearch version requirements
  • Ron Roskens worked on updating our build system to build using JDK 11
  • Chandra did more work on an OpenConfig integration
  • I continued to work on polishing up DJ's Giant Dev-Jam Test Refactor Branch
  • Dustin fixed an issue with overlapping RRAs in the Newts converter
  • Christian did some work on moving legacy node assets to meta-data
  • Dustin worked on some tools to validate our documentation build
  • Jeff worked on some tools to make interating with the Karaf shell easier
  • Christian made it so meta-data can be used in notifications
  • Chandra added support for the gRPC API to OIA
  • Patrick fixed an issue with numeric node labels in resource graph properties
  • Bonnie worked on wrapping up her changes to the user documentation
  • Patrick and Christian worked on the remote poller Minion refactor
  • Chandra fixed an issue with new suspect events sending even if an interface is invalid
Web, ReST, UI, and Helm
  • Jeff worked on adding some dashboards to Helm
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jeff Gehlbach
  • Jesse White
  • Patrick Schweizer
  • Ronald Roskens

Dev-Jam 2020: A Virtual Success

We'll be posting more about this over the coming days to the OpenNMS blog but we had a lot of fun doing virtual Dev-Jam.
Was it the same? No, of course not, but the whole Virtual Dev-Jam Committee did a great job of putting together a week of community fun.

The biggest thing about Virtual Dev-Jam compared to the real thing is that we wanted to get together in a way that felt like hanging out.
Jess spent some time looking through a bunch of virtual conferencing options, but in the end we decided to go with... Minecraft!
Zoë found us a fun map and set about turning it into a bunch of useful spaces with some help from her son, and a bunch of folks had a blast modifying things and making it our own.
You can see a map here.

Alongside Minecraft we used an open-source voice chat app called Mumble which has support for positional audio, and a plugin to link the two together.
That means we could have a virtual space where we could hear each other if we were close together, but could still split off to have conversations in separate spaces without talking over each other in global chat.
It ended up working out really well!

Tarus gave his usual Monday morning speech thanking everyone for being there, and then we goofed around in Minecraft for a while, then broke to work on our individual projects.

Tarus's POV During Morning Announcements

By the end of the week, folks had built an after-hours hangout complete with roller coaster, boating, and aquarium.

On the final afternoon the usual presentations were done, showing the progress everyone made over the week.
It was all streamed to Twitch and recorded, expect to see posts soon about some of the neat things people got working.

August Releases: Meridians 2017 through 2019 and Horizon 26.1.3

In August we released another round of bug fix releases, most notably Horizon 26.1.3 which included big optimizations to the new time-series layer, some UI fixes, and more.

Meridian 2019.1.10 got a number of those bug fixes as well, and Meridians 2018.1.21 and 2017.1.25 got a documentation update and a fix to the RRD-to-Newts converter.

Check out the release announcement for more details.

The OpenNMS Group, Inc. Acquired by NantHealth

As you may or may not have heard, The OpenNMS Group has been acquired by NantHealth. A full announcement is available on opennms.com but I wanted to talk about it a little bit here.

Since OpenNMS On the Horizon is really about the project -- not The OpenNMS Group -- I try not to focus too much on what we as a company do here other than Meridian releases, since it's not terribly relevant to the open source side of things.

TOG has always been happy to be a steward to the OpenNMS project, with support and sponsored feature development as our source of income to keep things running smoothly.

Now that this acquisition has happened, how does it change things for the OpenNMS project?
Not a whole lot.

OpenNMS, the project, will continue as it always has, with TOG as a primary contributor.
TOG, as a company, will get resources to also branch out into other things that we've been working on bit by bit behind the scenes for a long time, just faster.
NantHealth will get a world-class core for monitoring medical equipment and server infrastructure, and the ability to leverage the Minion and ALEC as tools to achieve those goals.

In the end we look forward to continuing to engage with the community as we always have, and to continue to develop one of the most flexible and powerful monitoring tools on the planet.

Thanks for going on this ride with us, now and into the future!

Calendar of Events

Upcoming Releases - September 1st, 2020

The next OpenNMS release day is September 1st, 2020.
Currently we expect new bugfix releases from Meridian 2019 and Horizon 26.

Also coming up next is Horizon 27.
While we can't yet say when it will be released, it's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware metadata has been moved from assets to the new node metadata
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
  • BMP improvements

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-12753: Support encryption for SNMP V3 credentials
  • NMS-12806: Rendering problems with complex custom Flow Classification Rules
  • NMS-12809: Update Users chapter
  • NMS-12820: Remote Poller: Modify model to new structure
  • NMS-12825: NPE while running AlarmLifecycleListenerManager
  • NMS-12827: Allow meta-data in notifications
  • NMS-12836: RRD to Newts converter fails with fully overlapping RRAs
  • NMS-12841: Remote Poller: camel case in new column definitions causing problems

by RangerRick at August 10, 2020 04:16 PM

August 05, 2020

August Updates for Meridian 2017 through 2019 and Horizon 26

OpenNMS is released on a monthly schedule, the first Tuesday of the month. The next scheduled release will be September 1st.

In August, we released updates for all supported OpenNMS releases.

Horizon 26.1.3

Release 26.1.3 is the sixth release in the Horizon 26 series.

It contains a bunch of bug fixes and improvements, including big optimizations to the new time-series layer, some UI fixes, and more.

The codename for 26.1.3 is Bit.

Bug Fixes
  • Using Netflow aggregations results in NPE when no results returned from ES (Issue NMS-12761)
  • Fix docs warnings for resource-types, time series config and thresholding (Issue NMS-12770)
  • interfaceSnmpByIfIndex fails if SNMP interface has no physical address (Issue NMS-12775)
  • Searching for alarms in the v2 API with a reductionKey that includes a comma or semicolon results in a 500 error (Issue NMS-12777)
  • Update log4j2 to 2.13.2 (Issue NMS-12787)
  • Support for optional snmpTrapAddress varbind needs documenting (Issue NMS-12795)
  • Broken link to "Standalone HTTPS with Jetty" in documentation. (Issue NMS-12804)
  • Rendering problems with complex custom Flow Classification Rules (Issue NMS-12806)
  • RRD-to-Newts Converter doesn't handle fully-overlapping RRAs (Issue NMS-12835)
  • RRD to Newts converter fails with fully overlapping RRAs (Issue NMS-12836)
Enhancements
  • Encrypt the password in REST API POST endpoint /opennms/rest/users (Issue NMS-6470)
  • Optimize Performance of Timeseries Integration Layer (Issue NMS-12731)
  • Limit number of calls to find metrics (Issue NMS-12744)
  • Limit calls to the timeseries_metatable (Issue NMS-12745)
  • Remove (if possible) the conversion from Opennms -> Newts -> TS (Issue NMS-12746)
  • Add JMX Meters to measure and export performance (Issue NMS-12747)
  • Remove unnecessary writes to meta_data table (Issue NMS-12748)
  • Remove conversion to Newts objects in read operation (Issue NMS-12749)
  • Clean up package structure (Issue NMS-12751)
  • Review documentation (Issue NMS-12758)
  • Remove nececessity to retrieve Metric when reading Samples (Issue NMS-12786)
  • Update OpenNMS DB functions and tests to handle Postgres 12 (Issue NMS-12819)

Meridian 2019.1.10

Release 2019.1.10 contains a few enhancements and a number of bugfixes.

The codename for 2019.1.10 is Ceres.

Bug Fixes
  • interfaceSnmpByIfIndex fails if SNMP interface has no physical address (Issue NMS-12775)
  • Searching for alarms in the v2 API with a reductionKey that includes a comma or semicolon results in a 500 error (Issue NMS-12777)
  • Backport log4j version update to older release(s) (Issue NMS-12791)
  • Support for optional snmpTrapAddress varbind needs documenting (Issue NMS-12795)
  • Broken link to "Standalone HTTPS with Jetty" in documentation. (Issue NMS-12804)
  • Rendering problems with complex custom Flow Classification Rules (Issue NMS-12806)
  • RRD-to-Newts Converter doesn't handle fully-overlapping RRAs (Issue NMS-12835)
  • RRD to Newts converter fails with fully overlapping RRAs (Issue NMS-12836)
Enhancements
  • Encrypt the password in REST API POST endpoint /opennms/rest/users (Issue NMS-6470)
  • Update OpenNMS DB functions and tests to handle Postgres 12 (Issue NMS-12819)

Meridian 2017.1.25 and 2018.1.21

Both the Meridian 2017 and 2018 releases have the same changes applied.

2017.1.25 has the codename "Mecca", and 2018.1.21 is codenamed "Blizzard."

Bug Fixes
  • Support for optional snmpTrapAddress varbind needs documenting (Issue NMS-12795)
  • RRD-to-Newts Converter doesn't handle fully-overlapping RRAs (Issue NMS-12835)
  • RRD to Newts converter fails with fully overlapping RRAs (Issue NMS-12836)

by RangerRick at August 05, 2020 02:42 PM

July 29, 2020

Tomorrow! OpenNMS Group at TM Forum’s Catalyst DIgital Showcase

OpenNMS Group will be participating in a session at TM Forum’s Catalyst Digital Showcase, along with partners Tech Mahindra and Cortex, on Thursday, July 30, 14:00-14:30, CEST.

Presented by the TM Forum, a global industry association, catalysts are rapid-fire proof-of-concept projects that bring together large and small companies to create innovative solutions to common challenges, leveraging key TM Forum best practices and standards.

The session will demonstrate the use of automation and orchestration technologies to transform communications service providers’ (CSP) business operations from being based on legacy API integrations to being based on TM Forum Network-as-a-Service (NaaS) API specifications, with a specific focus on the service problem management interactions between two CSPs.

OpenNMS has joined with Cortex and Tech Mahindra to demonstrate service provider integration using the TM Forum Service Problem Management API. As an open source project, OpenNMS has provided a flexible architecture to prototype solutions and explore NaaS use cases. Dr. Craig Gallen will explain how OpenNMS has been used in the first phase of the catalyst and the proposed next steps towards a second phase in October.

There’s still time to register.

by Bonnie at July 29, 2020 12:45 PM

July 27, 2020

OpenNMS On the Horizon – Bug Fixes, Remote Poller on Minion, Helm, DEV JAM!

In the last week we we fixed a bunch more bugs, did more work on moving the remote poller to the Minion, and picked up work on Helm again. This week: DEV JAM!

Github Project Updates

Internals, APIs, and Documentation
  • Justin Wood contributed a fix for handling interfaceSnmpByIfIndex on interfaces with no physaddr
  • I worked on fixing issues with startup and systemd and PID handling
  • Ronny did more fixes on certificate issues in our new Jetty updates
  • Christian and Patrick continued with the work on moving remote poller functionality to the Minion
  • Chandra fixed an issue with bad SNMP agents that return an unexpected number of oids
  • Sean fixed the InstallerDb to handle PostgreSQL 12 properly
  • Sean worked on some Kafka-related updates
  • Christian fixed an issue with report definition parsing
Web, ReST, UI, and Helm
  • I started working on some fixes for Helm in Grafana 6.7 and Grafana 7
  • I added a feature to hash passwords when submitting them in the users ReST interface
  • I fixed an issue displaying large rules in the correlator config UI
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Chandra Gorantla
  • Christian Pape
  • Justin Wood
  • Patrick Schweizer
  • Ronny Trommer
  • Sean Torres

Calendar of Events

Virtual Dev-Jam 2020 -- RIGHT NOW

This year we will, for obvious reasons, not be doing our normal yearly Dev-Jam hackathon.
With that in mind, we're going to try doing something completely different: Virtual Dev-Jam.

We are running a Minecraft server, with support for positional sound using Mumble, which means you can be there and hear audio only from the folks near you.

Also you can join the "Dev Jam" channel in the OpenNMS chat server.

We also plan on doing a movie screening, gaming, and other fun stuff.

For details, check out VIRTUAL DEVJAM.

Upcoming Releases - August 4th, 2020

The next OpenNMS release day is August 4th, 2020.
Currently we expect new bugfix releases from Meridian 2019 and Horizon 26.

Also coming up soon is Horizon 27, hopefully by early fall.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware metadata has been moved from assets to the new node metadata
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
  • BMP improvements

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • HELM-243: Cannot see list of nodes and resources when editing a panel.
  • NMS-6470: Encrypt the password in REST API POST endpoint /opennms/rest/users
  • NMS-10526: Trailing ", \" in report definitions throws not helpful error message
  • NMS-12775: interfaceSnmpByIfIndex fails if SNMP interface has no physical address
  • NMS-12777: Searching for alarms in the v2 API with a reductionKey that includes a comma or semicolon results in a 500 error
  • NMS-12813: Include XML schema for wsman-datacollection-config.xml in assemblies
  • NMS-12819: Update OpenNMS DB functions and tests to handle Postgres 12
  • NMS-12823: Application Topology Provider Status

by RangerRick at July 27, 2020 06:44 PM

July 20, 2020

OpenNMS On the Horizon – Documentation, OpenConfig gRPC, Startup, Integration API, SNMPv3, CircleCI, Remote Polling, Virtual Dev-Jam 2020

In the last week we worked on documentation, OpenConfig gRPC, OpenNMS startup, OIA, SNMPv3 credentials, CI, and moving remote polling to the Minion.

Github Project Updates

Internals, APIs, and Documentation
  • Chandra worked on an OpenConfig gRPC telemetry implementation
  • Ronny fixed up some formatting issues in documentation
  • I updated our Log4j2 to the latest version
  • I fixed a few issues with PID file handling, including writing a PID when run under systemd and moving karaf.pid back into $OPENNMS_HOME/logs/ after a rergression
  • Patrick worked on some OIA integration tests
  • Bonnie did a bunch more documentation updates including fixing the HTTPS Jetty docs, Grafana PDF reports, and user management
  • Chandra did a bit more work on storing SNMPv3 credentials securely
  • I worked on trying to switch our CI workflow to use the new Cloudsmith repos
  • Christian continued to work on polishing the new remote poller Minion functionality

Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Jeff Gehlbach
  • Patrick Schweizer
  • Ronny Trommer
  • Sean Torres

Calendar of Events

Upcoming Releases - August 4th, 2020

The next OpenNMS release day is August 4th, 2020.
Currently we expect new bugfix releases from Meridian 2019 and Horizon 26.

Also coming up soon is Horizon 27, hopefully by early fall.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware metadata has been moved from assets to the new node metadata
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
  • BMP improvements
Virtual Dev-Jam 2020

This year we will, for obvious reasons, not be doing our normal yearly Dev-Jam hackathon.
With that in mind, we're going to try doing something completely different: Virtual Dev-Jam.

We have a Minecraft server we'll be running, with support for positional sound using Mumble, which means you can be there and hear audio only from the folks near you.

We also plan on doing some other social stuff.

For details, check out VIRTUAL DEVJAM.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-12731: Optimize Performance of Timeseries Integration Layer
  • NMS-12770: Fix docs warnings for resource-types, time series config and thresholding
  • NMS-12787: Update log4j2 to 2.13.2
  • NMS-12791: Backport log4j version update to older release(s)
  • NMS-12795: Support for optional snmpTrapAddress varbind needs documenting
  • NMS-12796: Move integration test to OIA
  • NMS-12798: Profile Performance of TSS
  • NMS-12803: Missing image reference on Grafana PDF reports documentation
  • NMS-12804: Broken link to "Standalone HTTPS with Jetty" in documentation.
  • NMS-12807: Misnumbered table of contents

by RangerRick at July 20, 2020 04:41 PM

July 13, 2020

OpenNMS On the Horizon – SNMPv3, Dependencies, Kafka Certificates, Flows, CircleCI, Documentation

In the last week we did more changes for SNMPv3 credentials, updated dependencies, Kafka certificate validation, flows, continuous, and documentation.

Github Project Updates

Internals, APIs, and Documentation
  • Chandra worked on encrypting stored SNMPv3 credentials.
  • I did a number of dependency updates.
  • Chandra made Kafka certificate validation configurable.
  • Bonnie did more work on documentation for thresholding metadata, and users.
  • Sean worked on a fix for the flow "last seen" column not getting updated in raw mode.
  • Christian continued his work on moving remote polling functionality to the Minion.
  • Jeff added documentation for the snmpTrapAddress varbind, used when processing proxied SNMP traps.
  • I updated our CircleCI builds to perform some extra steps to publish documentation, XSDs, and other build output used during releases as artifacts.

Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Jeff Gehlbach
  • Patrick Schweizer
  • Sean Torres

Calendar of Events

Upcoming Releases - August 4th, 2020

The next OpenNMS release day is August 4th, 2020.
Currently we expect new bugfix releases from Meridian 2019 and Horizon 26.

Also coming up soon is Horizon 27, hopefully by early fall.
It's going to contain a bunch of great stuff:

  • improvements to node caching for flow processing
  • VMware metadata has been moved from assets to the new node metadata
  • Minion improvements, including configuration enhancements and an overhaul to RPC thread-handling
  • a new handy global search bar in the web UI
  • a major rework of remote poller functionality, now integrated with the Minion (we're calling it Application Perspective Monitoring)
  • BMP improvements

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • ALEC-88: Tick Errors in karaf.log that might be preventing ALEC to perform correlations.
  • NMS-12709: Document Resource types
  • NMS-12736: Add documentation for TcpListener
  • NMS-12744: Limit nubmer of calls to find metrics
  • NMS-12745: Limit calls to the timeseries_metatable
  • NMS-12746: Remove (if possible) the conversion from Opennms -> Newts -> TS
  • NMS-12747: Add JMX Meters to measure and export performance
  • NMS-12748: Remove unnecessary writes to meta_data table
  • NMS-12749: Remove conversion to Newts objects in read operation
  • NMS-12751: Clean up package structure
  • NMS-12758: Review documentation
  • NMS-12759: Optimize Performance of InfluxDb Plugin
  • NMS-12761: Using Netflow aggregations results in NPE when no results returned from ES
  • NMS-12771: Evaluate and improve opennms-cortex-tss-plugin
  • NMS-12776: Flow last seen db columns not being updated when raw records disabled
  • NMS-12785: RemotePoller: only schedule services that are part of an application

by RangerRick at July 13, 2020 04:31 PM

July 09, 2020

When Build Challenges Lead to Creative Solutions

A good build environment needs the right balance among complexity, speed, cost, and adaptability. With our recent move to CircleCI, OpenNMS has achieved that balance and even created two customized tools that others might find useful.

Complexity of a Large-Scale Project

At over 5 million lines of code, OpenNMS is one of the largest open source Java projects. With three flavors of OpenNMS — Horizon, Meridian, PoweredBy — there is a lot to keep track of.

In brief, our workflow sees bug fixes go into our oldest relevant branch. New features and enhancements go into the Horizon “release-NN.x” branch or “develop”. With automated continuous integration builds, we merge bug and security fixes made to older foundation branches forward:

Build Process Took Too Long

Before moving everything to CircleCI, builds were taking 20-30 minutes on a good machine, but running the test suite took an additional nine hours with the integration and smoke tests. The situation was becoming untenable: waiting for a build, then possibly adding a fix, then waiting again for a new build sometimes meant two days or more to get a green build.

Now, after migration, the entire build time, including testing, takes only 1.5 hours.

How We Did It

We took advantage of CircleCI’s parallel build feature to use multiple machines for the integration tests. This greatly speeds up the processing time, meaning that developers can build, test, fix, and rebuild within a few hours instead of days.

Tests are more self-contained. Previously, with Bamboo, creating new tests could take a week to get things set up. Now we have Dockerized our smoke tests so that they are repeatable without having to use crazy environments.

At this point, we have automated and moved anything significant that was in Bamboo into CircleCI. We will move other parts of the code base that don’t get updated very often on an as-needed basis.

Migration Challenges

The main challenge in migrating to CircleCI was the test part of the CircleCI workflow. For example, to test something that happens an hour into the process, you have to wait an hour to see if it worked. The longer into the process, the longer the wait.

Trying to figure out how to get code signing into the mix was also an issue, because you don't want to leave your private keys anywhere that automated code is accessing. The way to address this in CircleCI is to create an environment variable that contains a text-encoded version of your private key, and then when the build initializes, decode it and write it to disk.

Part of CircleCI's security setup involves making it so that these variables are not sent along to pull-request builds, so someone couldn't create a malicious pull request and extract our private keys.

To solve the code signing problem, we created an orb to sign packages.

Signing Packages with a Custom Orb

An orb is a shareable package of CircleCI configuration for use in builds. We created an orb that automates the process of turning those environment variables into Java jar-signing and GnuPG signing configs, and then also provides commands to perform the signing once you've created unsigned Debian and RPM packages.

This orb is available on the CircleCI website for anyone to use in their own CircleCI environments.

Disadvantages of the New Environment

Cost can be a disadvantage, as CircleCI server management can be expensive, as the cost of chaining multiple machines is a factor, particularly if we need to add more machines to handle the load.

But if you factor in the amount of time Ben Reed, Principal Software Developer and Build Manager at OpenNMS, spent “babysitting” the previous system, we are probably saving money.

“I feel like we are able to do just as much, if not more, with Circle than Bamboo,” Ben says. He adds that CircleCI is pretty easy to work with. “There is not a lot of figuring out how the environment works. It’s simple, but it can be built up to do other things that are more complex.”

What Does This Mean for OpenNMS Users?

Changes to the build environment may affect people like developers who do their own builds, and PoweredBy users who distribute OpenNMS under their own commercial End User’s License Agreement (EULA). PoweredBy users should read the PDF document in the PoweredBy repo that describes the steps needed to work with the new build environment.

Docker Improvements As Well

During the migration process, we also worked to cut down the size of the Horizon and Minion Docker containers. Each Docker command runs and saves snapshots of the file system. These layers build up, until we saw 500MB in Horizon and 200MB in Meridian and Sentinel that weren’t being used. We wrote a separate Docker container that runs a yum server that removes the files, so you never have the temporary files being saved.

Frustrating at Times, But in the End, Worth It

Ben estimates that the work was 50/50 between enjoyment and frustration. “It’s very gratifying when you’re done work and can see it all go through the flow,” he says. “It’s not the most enjoyable project, but finishing it delivered a great deal of satisfaction.”

by Bonnie at July 09, 2020 02:51 PM

July Updates for Meridian 2016 through 2019 and Horizon 26

OpenNMS is released on a monthly schedule, the first Tuesday of the month. The next scheduled release will be August 4th.

In July, we released updates to all Meridian versions from 2016 through 2019, as well as Horizon 26:

Horizon 26.1.2

Release 26.1.2 is the fifth release in the Horizon 26 series.

It contains a bunch of bug fixes and improvements, including docker container optimizations,
Netflow changes, documentation updates, time-series enhancements, and more.

The codename for 26.1.2 is Plague.

Bug Fixes
  • AbstractSnmpValue.allBytesDisplayable() IndexOutOfBound Exception (Issue NMS-7547)
  • Unable to collect SNMP through minions on a large scale (Issue NMS-10389)
  • Update examples/opennms.conf to be JDK11-compatible (Issue NMS-12468)
  • RRD-to-Newts converter only handles AVERAGE RRAs (Issue NMS-12722)
  • Parameters with dots handled incorrectly in BMP feature config (Issue NMS-12738)
  • The ReST end-point for the Flow Exporter details is returning invalid content (Issue NMS-12740)
  • Netflow 5 records in ES do not contain value for delta_switched (Issue NMS-12750)
  • dependency commons-beanutils 1.8.3 vulnerability (Issue NMS-12757)
  • Template field 'APPLICATION TAG' has illegal size (Issue NMS-12783)
  • Kafka Producer puts all events on the same partition when using donotpersist (Issue NMS-12784)
Enhancements
  • Reduce Docker container image size (Issue NMS-12284)
  • Document how to use meta-data with thresholding (Issue NMS-12735)
  • Add documentation for TcpListener (Issue NMS-12736)
  • upgrade to latest Jetty security/bug fixes (Issue NMS-12743)
  • Run a comparison: implementation before changes and after (Issue NMS-12752)
  • Optimize Performance of InfluxDb Plugin (Issue NMS-12759)
  • Be able to ignore certificate validation on all Karaf features that push data to Elasticsearch (Issue NMS-12768)
  • Evaluate and improve opennms-cortex-tss-plugin (Issue NMS-12771)
  • Provide a test harness for time series plugins (Issue NMS-12772)

Meridian 2019.1.9

Release 2019.1.9 is a small update to 2019.1.8 that fixes a few bugs and makes some Docker-related improvements.

BREAKING: This release changes the Systemd service name back from meridian to opennms to match previous releases. You may need to run systemctl disable meridian and/or systemctl enable opennms to make sure OpenNMS starts on reboot.

The codename for 2019.1.9 is Pluto.

Bug Fixes
  • AbstractSnmpValue.allBytesDisplayable() IndexOutOfBound Exception (Issue NMS-7547)
  • Update examples/opennms.conf to be JDK11-compatible (Issue NMS-12468)
  • RRD-to-Newts converter only handles AVERAGE RRAs (Issue NMS-12722)
  • dependency commons-beanutils 1.8.3 vulnerability (Issue NMS-12757)
  • Kafka Producer puts all events on the same partition when using donotpersist (Issue NMS-12784)
  • The Systemd service definition is called meridian not opennms (Issue LTS-239)
Enhancements
  • Reduce Docker container image size (Issue NMS-12284)
  • upgrade to latest Jetty security/bug fixes (Issue NMS-12743)

Meridian 2018.1.20

Release 2018.1.20 is a small update to 2018.1.19 that fixes a few bugs and makes some Docker-related improvements.

The codename for 2018.1.20 is Avalanche.

Bug Fixes
  • AbstractSnmpValue.allBytesDisplayable() IndexOutOfBound Exception (Issue NMS-7547)
  • Update examples/opennms.conf to be JDK11-compatible (Issue NMS-12468)
  • RRD-to-Newts converter only handles AVERAGE RRAs (Issue NMS-12722)
  • dependency commons-beanutils 1.8.3 vulnerability (Issue NMS-12757)
  • Kafka Producer puts all events on the same partition when using donotpersist (Issue NMS-12784)
Enhancements
  • Reduce Docker container image size (Issue NMS-12284)
  • upgrade to latest Jetty security/bug fixes (Issue NMS-12743)

Meridian 2017.1.24

Release 2017.1.24 is a small update to 2017.1.23 that fixes a few bugs and makes some Docker-related improvements.

The codename for 2017.1.24 is Kew meridian.

Bug Fixes
  • AbstractSnmpValue.allBytesDisplayable() IndexOutOfBound Exception (Issue NMS-7547)
  • RRD-to-Newts converter only handles AVERAGE RRAs (Issue NMS-12722)
  • dependency commons-beanutils 1.8.3 vulnerability (Issue NMS-12757)
Enhancements
  • Reduce Docker container image size (Issue NMS-12284)
  • Bump Docker base dependencies in build-env and OCI artifacts (Issue NMS-12699)
  • upgrade to latest Jetty security/bug fixes (Issue NMS-12743)

Meridian 2016.1.24

Release 2016.1.24 is a small update to 2016.1.23 that fixes a few bugs and makes some Docker-related improvements.

Meridian 2016 is technically out of support, but since there is a security fix for commons-beanutils it was deemed worth making new binaries.

The codename for 2016.1.24 is Breusing Geometric.

Bug Fixes
  • AbstractSnmpValue.allBytesDisplayable() IndexOutOfBound Exception (Issue NMS-7547)
  • Confd download fails silently on Docker install (Issue NMS-12642)
  • RRD-to-Newts converter only handles AVERAGE RRAs (Issue NMS-12722)
  • dependency commons-beanutils 1.8.3 vulnerability (Issue NMS-12757)
Enhancements
  • Reduce Docker container image size (Issue NMS-12284)
  • Backport CircleCI pipeline to foundation-2016 (Issue NMS-12607)
  • Bump Docker base dependencies in build-env and OCI artifacts (Issue NMS-12699)

by RangerRick at July 09, 2020 08:00 AM

July 06, 2020

OpenNMS On the Horizon – Bug Fixes, JDK Build System Updates, Encrypted SNMPv3, New Package Repositories

In the last week we did more bug fixing, worked on JDK build system updates, encrypted SNMPv3 credentials, new package repositories, and more.

Github Project Updates

Internals, APIs, and Documentation
  • Patrick continued his work on optimizing the new time-series APIs.
  • Marcel fixed an issue with parsing APPLICATION TAG in Netflow 9 packets.
  • Chandra worked on storing SNMPv3 credentials encrypted in the key-value store.
  • Ron Roskens continued his work updating our build system to work against newer JDKs.
  • Chandra added support for ignoring certificate validation on Karaf features pushing to Elasticsearch.
  • Chandra fixed an issue with Karaf partitioning and (donotpersist) events without an event ID.
  • Christian did more work on moving the remote poller functionality to Minion.

Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Chandra Gorantla
  • Christian Pape
  • Marcel Fuhrmann
  • Patrick Schweizer
  • Ronald Roskens

New Package Repositories

We are in the process of moving our package repositories from self-hosted and self-mirrored to using Cloudsmith.

They are live and active now, and CircleCI has been pushing to them for a few months, but we would like some folks to try them out and let us know if you run into any issues.

For more details on using the new repositories, see this post on our OpenNMS Discourse.

Calendar of Events

July Releases - July 7th, 2020

The next OpenNMS release day is July 7th, 2020. Currently we expect releases from all supported Meridians as well as Horizon 26, due to backported updates for the RRD Newts converter and an update to the default Java version used in Docker images.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • ALEC-84: Add Graph Provider to support ALEC
  • NMS-12468: Update examples/opennms.conf to be JDK11-compatible
  • NMS-12736: Add documentation for TcpListener
  • NMS-12760: Add ReST interface for Remote Poller
  • NMS-12768: Be able to ignore certificate validation on all Karaf features that push data to Elasticsearch
  • NMS-12783: Template field 'APPLICATION TAG' has illegal size
  • NMS-12784: Kafka Producer puts all events on the same partition when using donotpersist

by RangerRick at July 06, 2020 04:58 PM

June 29, 2020

OpenNMS On the Horizon – Build and Infrastructure, Documentation, Flow and Measurements Bug Fixes, and Minion Remote Poller

In the last week we did more build and infrastructure work, documentation updates, flow and measurements bugs, and the Minion-based remote poller.

Github Project Updates

Internals, APIs, and Documentation
  • Ron Roskens worked on resurrecting his Dev Jam work to build on a JDK newer than 1.8.
  • I did more work on infrastructure updates for CI and repo serving.
  • Bonnie did more work on thresholding metadata and TCP listener documentation.
  • Chandra worked on making the Minion-based remote poller handle poller package association.
  • Ronny fixed a bunch of Asciidoc documentation warnings.
  • Sean worked on a bug where "last seen" wasn't updating with flow raw records disabled.
  • Justin Wood submitted a fix for interfaceSnmpByIfIndex when an interface has no associated physaddr.
  • Marcel worked on a fix for handling the APPLICATION TAG field from Netflow 9.
  • Ronny worked on updating us to the latest 4.x JNA.
Web, ReST, UI, and Helm
  • Christian added a ReST interface for accessing Minion remote poller data.
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Justin Wood
  • Marcel Fuhrmann
  • Matthew Brooks
  • Patrick Schweizer
  • Ronald Roskens
  • Ronny Trommer
  • Sean Torres

Calendar of Events

July Releases - July 7th, 2020

The next OpenNMS release day is July 7th, 2020. Currently we expect releases from all supported Meridians as well as Horizon 26, due to backported updates for the RRD Newts converter and an update to the default Java version used in Docker images.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • ALEC-85: ALEC smoke tests currently failing loading in Sentinel
  • NMS-12677: Dynamic service scheduling: Polling package association
  • NMS-12735: Document how to use meta-data with thresholding
  • NMS-12752: Run a comparison: implementation before changes and after
  • NMS-12757: dependency commons-beanutils 1.8.3 vulnerability
  • NMS-12770: Fix docs warnings for resource-types, time series config and thresholding
  • NMS-12772: Provide a test harness for time series plugins
  • NMS-12781: Migrated VMware asset data to metadata entries
  • ORG-55: http://xmlns.opennms.org content no longer available
  • ORG-87: xmlns went missing as part of our office server shutdown

by RangerRick at June 29, 2020 04:04 PM

June 23, 2020

OpenNMS On the Horizon – Time-Series API, Minion Remote Poller, CircleCI, Netflow, VMware, and Documentation

In the last 2 weeks we did a lot of infrastructure and bug fixing, as well as continuing to work on optimizing the new time-series API.

Github Project Updates

Internals, APIs, and Documentation
  • Patrick continued his work on making performance improvements to the new time-series API.
  • Christian updated the new Minion-based remote poller to support thresholding on response times.
  • Bonnie added documentation for using metadata with thresholding.
  • I did a bunch of work on CircleCI improvements, including merging the fixes to shrink our Docker images.
  • Chandra fixed the reporting of delta_switched in Netflow 5 Elasticsearch records.
  • I updated our commons-beanutils dependencies to the latest version.
  • Jeff updated the admin guide with a Minion RPC troubleshooting example.
  • Christian did an update to move the VMware asset data to node metadata.
  • Sean fixed a bug in returning Netflow aggregations when no results are returned by Elasticsearch.
  • Bonnie did a bunch of cleanups in the developer documentation.
Web, ReST, UI, and Helm
  • Bonnie worked on Helm documentation for expressions and filtering.
  • I upgraded our Jetty to the latest 9.4.29.
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Jeff Gehlbach
  • Matthew Brooks
  • Patrick Schweizer
  • Sean Torres
  • Seth Leger

Calendar of Events

July Releases - July 7th, 2020

The next OpenNMS release day is July 7th, 2020. Currently we expect releases from all supported Meridians as well as Horizon 26, due to backported updates for the RRD Newts converter and an update to the default Java version used in Docker images.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • HELM-231: "How to configure the data sources in Grafana" docs are missing
  • HELM-240: Enhance HELM documentation
  • HELM-241: Add expression examples
  • HELM-242: JEXL expressions
  • NMS-7547: AbstractSnmpValue.allBytesDisplayable() IndexOutOfBound Exception
  • NMS-10389: Unable to collect SNMP through minions on a large scale
  • NMS-12284: Reduce Docker container image size
  • NMS-12682: Update collectors chapter
  • NMS-12721: Apply thresholding for remote poller response times
  • NMS-12738: Parameters with dots handled incorrectly in BMP feature config
  • NMS-12742: remove the java ("windows") installer
  • NMS-12750: Netflow 5 records in ES do not contain value for delta_switched

by RangerRick at June 23, 2020 08:46 PM

June 08, 2020

OpenNMS On the Horizon – Bug Fixes, Docker, Minion, Time-Series, Node Metadata, Documentation

In the last week we did the June releases, fixed some bugs, did more work on Docker impovements, continued to work on Minion updates, time-series enhancements, metadata improvements and documentation, flow ReST, and more.

Github Project Updates

Internals, APIs, and Documentation
  • We ressurected an old attempt to fix UTF-8 handling in SNMP values from Seth.
  • I continued to work on the changes to make docker images built from RPMs smaller.
  • Dustin did more work on updating the Remote Poller to use modern APIs and Minion.
  • Dustin fixed an issue in the RRD to Newts converter when step-size isn't 300.
  • Chandra continued his work on improving thread handling in the Minion.
  • Bonnie worked on documentation for metadata thresholding.
  • Patrick made improvements to ResourceDAO performance in the new time-series API.
  • Zoë fixed some issues in BMP telemetry when handling parameters with . in them.
  • I removed Windows-related installation references from the documentation for Horizon 27.
  • I fixed some issues with CircleCI automatic branch merging.
  • Christian worked on moving VMware asset metadata from node assets to the new node metadata API.
  • Patrick updated OIA to offer up tag names as constants in the time-series APIs.
Web, ReST, UI, and Helm
  • Sean fixed an issue with ingress/egress handling in the flow exporter ReST interface.
  • I worked on bumping our Jetty to the latest version.
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Mark Mahacek
  • Patrick Schweizer
  • Sean Torres
  • Seth Leger
  • Zoë Knox

June Releases - Meridian 2019.1.8, Horizon 26.1.1

For the June releases, we put out relatively small updates to Horizon 26 and Meridian 2019.

Both contained a number of fixes and a few enhancements, including Kafka event forwarding fixes. Horizon 26.1.1 also included a couple other bug fixes, most notably a fix for a performance regression in Netflow handling.

For more details, see the June release announcement.

Calendar of Events

July Releases - July 7th, 2020

The next OpenNMS release day is July 7th, 2020. Currently we expect releases from all supported Meridians as well as Horizon 26, due to backported updates for the RRD Newts converter and an update to the default Java version used in Docker images.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-3672: Poller Packages ignores include/excludes for remote polling
  • NMS-4671: JNA ping code returns timeouts for all link-local IPv6 addresses
  • NMS-4912: JdbcCollector freeze Collectd when using Data Source Factories defined on opennms-datasources.xml instead of using their own connections.
  • NMS-5628: Requisition File is Unmarshalled for each Node as it is Scanned
  • NMS-12678: Dynamic service scheduling: Poller-Configuration change
  • NMS-12698: Use CollectionSetBuilder to persist response times
  • NMS-12722: RRD-to-Newts converter only handles AVERAGE RRAs
  • NMS-12730: Meta-data tag enhancements to Time Series Storage API
  • NMS-12740: The ReST end-point for the Flow Exporter details is returning invalid content
  • NMS-12743: upgrade to latest Jetty security/bug fixes
  • OIA-26: Expose intrinsic tag names as constants for TSS API

by RangerRick at June 08, 2020 04:03 PM

June 02, 2020

June Updates for Meridian 2019 and Horizon 26

OpenNMS is released on a monthly schedule, the first Tuesday of the month. The next scheduled release will be July 7th.

In June, we released updates to Meridian 2019 and Horizon 26:

Meridian 2019.1.8

Release 2019.1.8 is the ninth release in the Meridian 2019 series.

It contains bug and documentation fixes as well as a few small enhancements.

The codename for 2019.1.8 is Neptune.

Bug Fixes
  • SSLCertMonitor server-name parameter results in NPE (Issue NMS-12332)
  • Fix warnings during documentation build (Issue NMS-12702)
  • Images are broken in admin guide (Issue NMS-12713)
  • Cleanup removed Elasticsearch REST plugin and hint to Plugin Manager (Issue NMS-12716)
  • Events forwarded by Kafka Producer doesn't have any parameters set (Issue NMS-12723)
Enhancements
  • Bump Docker base dependencies in build-env and OCI artifacts (Issue NMS-12699)
  • Send trouble ticket id to kafka alarm topic (Issue NMS-12725)

Horizon 26.1.1

Release 26.1.1 is the fourth release in the Horizon 26 series.

It contains documentation and bug fixes including improvements to Kafka event and alarm publishing.

The codename for 26.1.1 is Hydrating.

Bug Fixes
  • SSLCertMonitor server-name parameter results in NPE (Issue NMS-12332)
  • Provisiond accepts multiple primary SNMP interfaces (Issue NMS-12605)
  • Fix warnings during documentation build (Issue NMS-12702)
  • Cleanup removed Elasticsearch REST plugin and hint to Plugin Manager (Issue NMS-12716)
  • The Alarm History feature is not working (Issue NMS-12718)
  • Events forwarded by Kafka Producer doesn't have any parameters set (Issue NMS-12723)
  • Netflow ingress performance regression (Issue NMS-12724)
Enhancements
  • Bump Docker base dependencies in build-env and OCI artifacts (Issue NMS-12699)
  • Send trouble ticket ID to Kafka alarm topic (Issue NMS-12725)

by RangerRick at June 02, 2020 08:51 PM

June 01, 2020

OpenNMS On the Horizon – CircleCI, Docker, Minion, Time-Series and Kafka Metadata, and the Remote Poller

In the last week we did more work on CI and Docker images, Minion bugfixes, time-series metadata, refactoring the remote poller, and Kafka metadata improvements.

Github Project Updates

Internals, APIs, and Documentation
  • I worked on shrinking our Docker images.
  • Chandra continued to work on improving the Minion's thread usage.
  • Christian continued to work on refactoring the remote poller to be a part of the Minion.
  • Patrick and Jesse worked on improved metadata in the new time-series API.
  • Dustin fixed some bugs in the Newts RRD converter.
  • I did more work migrating Bamboo workflows to CircleCI.
  • Chandra enhanced forwarded Kafka alarms to include trouble-ticket metadata.

Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jesse White
  • Patrick Schweizer

Calendar of Events

June Releases - June 2nd, 2020

The next OpenNMS release day is June 2nd, 2020. Currently we expect point/bug fix releases of Meridian 2019 and Horizon 26.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-12391: Limit RPC threads on Minion using bulkhead pattern
  • NMS-12584: Create lab for simulating and persisting large volumes of flows
  • NMS-12724: Netflow ingress performance regression
  • NMS-12725: Send trouble ticket id to kafka alarm topic

by RangerRick at June 01, 2020 03:18 PM

May 26, 2020

OpenNMS On the Horizon – CircleCI, Kafka, Time-Series API, Netflow, Minion, OIA

In the last week we kept working on CI improvements, fixed some Kafka bugs, fixed time-series and netflow issues, the new Minion-based remote poller infrastructure, OIA integration updates, and more!

Github Project Updates

Internals, APIs, and Documentation
  • I did more work cleaning up and enhancing our CircleCI builds.
  • Chandra fixed an issue with the alarm history elasticsearch feature when indexPrefix is set.
  • Dustin did more work on modernizing the remote poller infrastructure.
  • Pierre added new confd templates for Minion to make SSH, RMI registry, and RMI server configurable.
  • Patrick did more work on fixing an issue when multiple SNMP interfaces are marked "primary" on a node.
  • Chandra continued his work on improving thread-handling on the Minion.
  • Chandra fixed an issue with missing parameters when forwarding events from the Kafka producer.
  • Jesse fixed a regression in throughput processing of Netflow ingress packets in Horizon 26.
  • Jesse fixed an issue with exceptions when persisting fails in the new time-series integration layer.
  • Matt did more work on storing passwords encrypted on-disk.
  • Patrick updated the OIA to expose tag names as constants accessible by integration implementers.
  • Patrick updated the time-series integration layer to pass improved metadata tags.
  • Ronny and I worked on some CircleCI changes that would reduce image sizes.

Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jesse White
  • Matthew Brooks
  • Patrick Schweizer
  • Pierre Bouffard
  • Ronny Trommer

Calendar of Events

June Releases - June 2nd, 2020

The next OpenNMS release day is June 2nd, 2020. Currently we expect point/bug fix releases of Meridian 2019 and Horizon 26.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • ALEC-67: Log message for situation reflects cleared state
  • ALEC-74: Kafka streams dies and does not recover from error
  • ALEC-84: Add Graph Provider to support ALEC
  • NMS-12332: SSLCertMonitor server-name parameter results in NPE
  • NMS-12605: Provisiond accepts multiple primary SNMP interfaces
  • NMS-12679: Update database and send events only on status changes
  • NMS-12687: Confd templates for Minion configuration (Karaf)
  • NMS-12699: Bump Docker base dependencies in build-env and OCI artifacts
  • NMS-12710: Create SQL index for optimizing poll result lookups
  • NMS-12718: The Alarm History feature is not working
  • NMS-12723: Events forwarded by Kafka Producer doesn't have any parameters set

by RangerRick at May 26, 2020 06:22 PM

May 21, 2020

Use Case: Monitoring Websites Using Metadata

OpenNMS’ Metadata DSL (domain specific language) allows you to use dynamic configuration in parameter values to interpolate metadata into the parameter. The syntax allows for the use of patterns in an expression, whereby the metadata is replaced with a corresponding value during the collection process.

You can use this feature with provisioning (service detectors), service assurance (pollerd), and performance management (collectd) in OpenNMS. Using metadata in these situations can streamline the configuration of things that require customization and items, such as protocol port numbers, that are unique to a node.

Ronny Trommer and Marcel Fuhrmann have written a use case that shows how to use metadata to monitor websites. The use case presents a scenario of running a blackbox test on a web server that hosts three websites, and a detailed explanation of how to configure OpenNMS to do this.

Link: Monitoring Websites Using Metadata

by Bonnie at May 21, 2020 02:26 PM

May 18, 2020

OpenNMS On the Horizon – Documentation Cleanup, CI Improvements, Bug Fixes, JDK Updates

In the last week we did more documentation cleanup, did more work on our continuous integration infrastructure, fixed some bugs, did some JDK11 build updates, and more!

Github Project Updates

Internals, APIs, and Documentation
  • I did more work on cleaning up and refactoring our CircleCI builds.
  • Ronny updated our docker builds to use a newer JDK.
  • Chandra worked on improving threading on the Minion.
  • Bonnie made some improvements to the Collectd documentation.
  • Ronny cleaned up some warnings in the documentation build.
  • Sean updated Kafka components to 2.5.0.
  • Christian did more work on refactoring the Minion to handle the remote poller's duties.
  • Chandra worked a bit more on making sure Provisiond doesn't allow multiple interfaces on a node to be marked "primary."
  • Chandra made it so OIA can be built with JDK 11.
  • Ronny cleaned up old references to plugins in the documentation.
  • I did some work on making our Docker images smaller.

Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Ronny Trommer
  • Sean Torres

Calendar of Events

June Releases - June 2nd, 2020

The next OpenNMS release day is June 2nd, 2020. Currently we expect point/bug fix releases of Meridian 2019 and Horizon 26.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • HELM-221: Make Helm docs publicly available
  • NMS-12683: Get new remote poller running as daemon
  • NMS-12702: Fix warnings during documentation build
  • NMS-12705: Release note changes for breaking changes are missing from Meridian
  • NMS-12713: Images are broken in admin guide
  • NMS-12715: Add very basic search capabilities to our docs framework
  • NMS-12716: Cleanup removed Elasticsearch REST plugin and hint to Plugin Manager

by RangerRick at May 18, 2020 07:27 PM

May 12, 2020

Recent Security Fix and Our Security Process

The OpenNMS Group recently learned about and fixed a security vulnerability that allowed remote code execution. CVE-2020-12760 applies to OpenNMS Horizon before 26.0.1, and Meridian before 2018.1.19 and 2019 before 2019.1.7. We advise customers to review the CVE and upgrade to the latest versions.

No one wants to have a security vulnerability, particularly with network management software, where the consequences could be serious. Through careful development, robust testing, and community and corporate vigilance, The OpenNMS Group does its best to produce secure software our users can trust. When someone notifies us about a security issue, we take action quickly to determine the severity, address it, and notify users.

1. Discovery

There are several ways we might learn about a security issue: from a community user, from a customer who finds something during their own routine internal IT security scanning processes, or through security researchers who actively look for security vulnerabilities in software and hardware. Contacting us through security@opennms.com sends a message to our security team, made up of key people from our development, customer support, and executive teams.

2. Responsible Disclosure

When we receive a notification about a possible security vulnerability, we immediately reach out to the reporter, using secure messaging, including PGP when working with researchers, if they wish. We thank them for notifying us, and particularly for choosing to disclose this information responsibly.

Dealing with a security vulnerability is a race to address it before it becomes widely known and exploitable. It requires balancing the ability to fix the vulnerability before someone exploits it and the risk to customers until that vulnerability is fixed.

For this reason, we appreciate responsible disclosure, which basically means the person who discovered the vulnerability does not publicize it until we have a chance to deal with it. In return, we take action immediately, and are transparent with the person who reported the issue, to keep them aware of our progress. In the case of CVE-2020-12760, we thank Florian Hauser, who works for Code White, for notifying us of this issue.

3. Verify Proof of Concept

Reporters usually provide us with a proof of concept (PoC) of how to exploit the problem they found and the steps to reproduce it. As mentioned earlier, CVE-2020-12760 allowed remote code execution.

The engineering team validates the PoC code, to determine the nature of the vulnerability and if it is reproducible. In some cases, the reporter may have been using an old version of the software, in which case we may have seen the issue already and fixed it, and it does not impact new versions of the software. We thank the reporter, and keep a bug report of the issue. Researchers usually work against the latest version, but sometimes they might use the most widely deployed version. Regardless, we take these reports seriously and investigate them.

4. Triage and Work

Once we verify the PoC, we determine the severity of the vulnerability, and create a JIRA issue visible only to core OpenNMS contributors. This keeps the issue hidden from searches until the bug is fixed to minimize the chance of someone exploiting the vulnerability. Once fixed, we change the permissions on the JIRA issue so that it becomes public and people can see a record of how it was fixed.

In serious and wide-spread cases, we inform Support and Meridian customers who might be exposed about the vulnerability, that we are addressing it, and to be prepared to install a fix when it becomes available. In some cases, there is a temporary workaround that we tell them to deploy until a fix is ready.

We update the reporter regularly on our progress with the issue, including PDF exports of the JIRA issue history. If they have an ID on our system, we add them to the permissions so they can follow the JIRA ticket.

5. Fix and Testing

All our work is done in GitHub. When we have a fix, we test it by running the PoC attack and verifying that the issue has been addressed, iterating as necessary until we have fixed the vulnerability. The researcher has the opportunity to test the fix, and once validated, we issue a new release and announcement about the vulnerability and fix.

6. Declaration and Communicating the Fix

When the fix is ready, we roll it out as soon as possible, with the next software release, or in an off-schedule release. We include information about the vulnerability in the release notes, social media, and blog posts, as well as notifying customers who are exposed. We make the JIRA issue public, and we credit the person who discovered the issue in these communications, unless they request not to be identified.

With severe issues we may create a CVE (Common Vulnerabilities and Exposures) declaration, that appears on a publicly maintained database of known exploitable vulnerabilities. This allows people to search for and discover OpenNMS security vulnerabilities and act accordingly to secure their system if they are affected. We also inform the reporter about the CVE.

We’re Never Done with Security

The world is full of “bad guys” ready to take advantage of people and their weaknesses, and of course, the Internet is no different. We must always be vigilant to potential threats, and think like the bad guys to stay ahead of them.

Building robust security testing processes into development, and using the latest tools and technologies (like OWASP, GitHub’s Dependabot, and SonarCloud, for example) to identify dependencies and fix vulnerabilities before release, are just some of the ways we can do this. Responsible disclosure and the efforts of security researchers, customers, and the OpenNMS community, are an essential part of the effort to help keep our products—and your business—secure.

by Bonnie at May 12, 2020 02:50 PM

May 11, 2020

OpenNMS On the Horizon – Releases, Documentation, Remote Poller, RPC, Karaf

In the last week we wrapped up some stuff for the releases, improved more documentation, continued to refactor the remote poller, made some RPC and Karaf improvements, and more.

Github Project Updates

Internals, APIs, and Documentation
  • I wrapped up my work on support for a simple plaintext Graphite telemetry adapter.
  • Chandra fixed an issue with multiple nodeDown messages on Pollerd reload.
  • Bonnie added documentation on generating PDFs from Grafana dashboards.
  • Bonnie worked on docs for the Prometheus collector.
  • Jesse released a new version of OIA to be included in Horizon 26.1.0.
  • Bonnie and Ronny worked on documenting restarting subsystems and thresholding.
  • Sean worked on updates to Elasticsearch used in unit tests.
  • Chandra fixed an issue with streaming telemetry and JDK 11.
  • Pierre did more work on adding more confd options to the Minion.
  • Marcel and Ronny updated the documentation for the SystemExecuteMonitor.
  • Dustin continued to work on refactoring remote poller functionality to the Minion.
  • Sean updated the Kafka components to version 2.5.0.
  • Sean worked on making HealthCheck support doing a subset of checks.
  • Chandra updated the Minion RPC to avoid creating too many threads.
  • Matt added support for encrypting password storage in the Karaf container.
Web, ReST, UI, and Helm
  • Patrick worked on tooltip support for the legacy graph provider.
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jeff Gehlbach
  • Jesse White
  • Marcel Fuhrmann
  • Matthew Brooks
  • Patrick Schweizer
  • Pierre Bouffard
  • Ronny Trommer
  • Sean Torres

May Releases

May release day was the 5th, and it brought updates to both Horizon and Meridian.

Meridian 2017.1.23, 2018.1.19, and 2019.1.7

We introduced new Meridian versions for 2017 through 2019, mostly with minor bug fixes and a few documentation enhancements in 2019.

Horizon 26.1.0

Horizon got a more substantial update, with bug fixes, a Prometheus collector, Graphite listener support, an Influx timeseries implementation, and a whole bunch of flow and other enhancements.

Calendar of Events

June Releases - June 2nd, 2020

The next OpenNMS release day is June 2nd, 2020. Currently we expect point/bug fix releases of Meridian 2019 and Horizon 26.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • HELM-237: Flow deep dive dashboard shows interfaces which don't belong to the selected node
  • NMS-9581: Document JDBCQueryMonitor "compare_string" Action
  • NMS-9804: Documentation: Install Guide: Compatibility Matrix Outdated
  • NMS-12269: Netscaler vServer SNMP collection and graph definitions
  • NMS-12568: Add an example for SystemExecuteMonitor into the docs
  • NMS-12647: Update smoketests to support various Kafka compression codecs
  • NMS-12650: Provide written procedures on the proper way to restart
  • NMS-12681: Reloading the Pollerd daemon causes multiple nodeDown messages
  • NMS-12684: Add upgrade task to enable RemotePollerNG
  • NMS-12688: Streaming Telemetry is broken when using OpenJDK 11 and minion
  • NMS-12694: Add tooltip support to the LegacyGraphProvider
  • NMS-12695: add a telemetry adapter for the Graphite plaintext protocol

by RangerRick at May 11, 2020 05:42 PM

May 07, 2020

New: Prometheus Collector

If you are using the Prometheus toolkit to monitor your network systems, you can now integrate that performance data into OpenNMS with the new PrometheusCollector.

The OpenNMS PrometheusCollector collects performance metrics via HTTP(S) using the text-based Prometheus Exposition format. Many applications have adopted this format, which the OpenMetrics project is in the process of standardizing.

How It Works

OpenNMS collects performance data using the Collectd daemon. Collectd schedules data collection on OpenNMS entities (currently nodes and interfaces) using management agents and protocol-specific collectors to collect performance metrics. Each collector has its own associated configuration that defines the parameters for the collector. (For more details on performance data collection in OpenNMS, refer to the product documentation.)

The PrometheusCollector parses and maps Prometheus-collected performance data into OpenNMS’s collection model. Manage Prometheus collection definitions in etc/prometheus-datacollection.d/, and set collection parameters in the /etc/collectd-configuration.xml file. See the PrometheusCollector documentation for more details.

Getting Started

By default we provide a sample configuration for Prometheus Node Exporter. The Node Exporter is an agent running on a server and provides detailed host machine metrics for Prometheus. Our sample configuration provides a starting point on how to use the PrometheusCollector with OpenNMS.

by Bonnie at May 07, 2020 06:42 PM

May 06, 2020

Updates for OpenNMS Horizon 26, Meridian 2019, 2018, and 2017 Released

OpenNMS Horizon 26.1.0 (Surgical) Released

Release 26.1.0 is the third release in the Horizon 26 series.

It is an enhancement release with a number of bug fixes and improvements, including updates to telemetry, provisioning, and more.

The codename for 26.1.0 is Surgical.

Bug
  • Security vulnerability in io.netty:netty-handler < 4.1.45 (need upgrade) (Issue NMS-12541)
  • NPE in KafkaFlowForwarder (Issue NMS-12660)
  • Add more context to Response Time resources (Kafka Producer) (Issue NMS-12661)
  • BMP parse error for path attribute MP_UNREACH_NLRI (Issue NMS-12671)
  • Reloading the Pollerd daemon causes multiple nodeDown messages (Issue NMS-12681)
  • Streaming Telemetry is broken when using OpenJDK 11 and minion (Issue NMS-12688)
Enhancement
  • Document JDBCQueryMonitor "compare_string" Action (Issue NMS-9581)
  • Add opentracing support for Provisiond (Issue NMS-12374)
  • SystemExecuteMonitor fails with exit code 6 (Issue NMS-12564)
  • Add an example for SystemExecuteMonitor into the docs (Issue NMS-12568)
  • Prometheus collector (Issue NMS-12577)
  • Timeseries Plugin Influx 1.x (Issue NMS-12633)
  • Update smoketests to support various Kafka compression codecs (Issue NMS-12647)
  • Bump ES version used in Smoke Tests (Issue NMS-12648)
  • Provide written procedures on the proper way to restart (Issue NMS-12650)
  • Aggregate flow metrics w/ stream processing (Issue NMS-12656)
  • Provisiond: Add NodeScanStarted event for scheduled scans (Issue NMS-12658)
  • Flow aggregation - alternate indices based on duration of time range filter (Issue NMS-12663)
  • Flow aggregation - Identify minimal set of fields required for current queries (Issue NMS-12664)
  • Enable node enrichment for Topology providers comming from the Integration Api (Issue NMS-12674)
  • Add tooltip support to the LegacyGraphProvider (Issue NMS-12694)
  • add a telemetry adapter for the Graphite plaintext protocol (Issue NMS-12695)

OpenNMS Meridian 2019.1.7 (Uranus) Released

Release 2019.1.7 is the eighth release in the Meridian 2019 series.

It contains a few bug fixes as well as a number of enhancements to the documentation.

The codename for 2019.1.7 is Uranus.

Bug
  • Prevent multiple node scans from being scheduled for a single node (Issue NMS-12504)
  • Add more context to Response Time resources (Kafka Producer) (Issue NMS-12661)
  • Reloading the Pollerd daemon causes multiple nodeDown messages (Issue NMS-12681)
  • Streaming Telemetry is broken when using OpenJDK 11 and minion (Issue NMS-12688)
Enhancement
  • Document JDBCQueryMonitor "compare_string" Action (Issue NMS-9581)
  • SystemExecuteMonitor fails with exit code 6 (Issue NMS-12564)
  • Add an example for SystemExecuteMonitor into the docs (Issue NMS-12568)
  • Provide written procedures on the proper way to restart (Issue NMS-12650)

OpenNMS Meridian 2018.1.19 (Sinkhole) Released

Release 2018.1.19 is a small release that fixes a couple of bugs.

The codename for 2018.1.19 is Sinkhole.

Bug
  • Reloading the Pollerd daemon causes multiple nodeDown messages (Issue NMS-12681)
Enhancement
  • SystemExecuteMonitor fails with exit code 6 (Issue NMS-12564)
  • Add an example for SystemExecuteMonitor into the docs (Issue NMS-12568)

OpenNMS Meridian 2017.1.23 (New Naval Observatory) Released

Release 2017.1.23 is a small update to 2017.1.22 that includes some build system updates and a fix for a bug when reloading Pollerd.

The codename for 2017.1.23 is New Naval Observatory meridian.

Bug
  • Reloading the Pollerd daemon causes multiple nodeDown messages (Issue NMS-12681

by RangerRick at May 06, 2020 06:15 PM

May 05, 2020

OpenNMS Meridian 2019.1.7 (Uranus) Released

Release 2019.1.7 is the eighth release in the Meridian 2019 series.

It contains a few bug fixes as well as a number of enhancements to the documentation.

The codename for 2019.1.7 is Uranus.

Bug

  • Prevent multiple node scans from being scheduled for a single node (Issue NMS-12504)
  • Add more context to Response Time resources (Kafka Producer) (Issue NMS-12661)
  • Reloading the Pollerd daemon causes multiple nodeDown messages (Issue NMS-12681)
  • Streaming Telemetry is broken when using OpenJDK 11 and minion (Issue NMS-12688)

Enhancement

  • Document JDBCQueryMonitor "compare_string" Action (Issue NMS-9581)
  • SystemExecuteMonitor fails with exit code 6 (Issue NMS-12564)
  • Add an example for SystemExecuteMonitor into the docs (Issue NMS-12568)
  • Provide written procedures on the proper way to restart (Issue NMS-12650)

by RangerRick at May 05, 2020 08:28 PM

OpenNMS Meridian 2018.1.19 (Sinkhole) Released

Release 2018.1.19 is a small release that fixes a couple of bugs.

The codename for 2018.1.18 is Sinkhole.

Bug

  • Reloading the Pollerd daemon causes multiple nodeDown messages (Issue NMS-12681)

Enhancement

  • SystemExecuteMonitor fails with exit code 6 (Issue NMS-12564)
  • Add an example for SystemExecuteMonitor into the docs (Issue NMS-12568)

by RangerRick at May 05, 2020 08:26 PM

OpenNMS Meridian 2017.1.23 (New Naval Observatory) Released

Release 2017.1.23 is a small update to 2017.1.22 that includes some build system updates and a fix for a bug when reloading Pollerd.

The codename for 2017.1.23 is New Naval Observatory meridian.

Bug

  • Reloading the Pollerd daemon causes multiple nodeDown messages (Issue NMS-12681

by RangerRick at May 05, 2020 08:24 PM

DYK: Generate PDF Reports of Grafana Dashboards with OpenNMS

OpenNMS makes it easy to generate PDF reports from Grafana dashboards. You can also schedule and email these PDF reports to anyone:

  • Keep staff without access to OpenNMS informed about network performance for improved capacity planning.
  • Create a permanent, historic record of strategic information by retaining reports in your inbox and/or the OpenNMS reporting library.
  • Create reports from any Grafana dashboard whether or not it has OpenNMS data.

How It Works

All you need is OpenNMS and an instance of Grafana with at least one dashboard and panel. OpenNMS allows you to create a PDF report for ANY Grafana dashboard, not just dashboards created using OpenNMS Helm. Simply obtain the Grafana Viewer API key from your Grafana instance, plug it into the Grafana endpoint configuration in OpenNMS, and you’re ready to go.

Set It and Forget It

With three templates (layouts) to choose from, your PDF can display one panel, two panels, or four panels per page. The templates have placeholders for each panel found on a Grafana dashboard.

OpenNMS connects with Grafana through the API key, retrieves a list of available dashboards, and displays them in the Report Parameters dialog:

Select a Grafana dashboard from the drop-down list, specify the time period for the report, and create a schedule to email the report to the people who need and want to see it.

From Dashboard to PDF:

Learn More!

For detailed procedures on how to create PDF reports from Grafana dashboards with OpenNMS, see the online documentation or check out this video.

by Bonnie at May 05, 2020 07:29 PM

May 04, 2020

OpenNMS On the Horizon – May 4th, 2020 – Documentation, Provisioning, Telemetry, Topology, Prometheus, Graphite, and More!

It's time for OpenNMS On the Horizon! (Did I miss last week? Shhh! Too busy doing cool stuff. Probably.)

In the last week we did a lot of plumbing work in Provisiond, Telemetryd, and topology, as well as more documentation improvements, a Prometheus collector, and a Graphite telemetry adapter.

Github Project Updates

Internals, APIs, and Documentation
  • I did more work fixing up CircleCI workflows.
  • Bonnie did more work on beefing up core documentation.
  • Jesse fixed an issue with serialized objects and ActiveMQ.
  • Chandra worked on enhancing the metadata provided with response time resources in Kafka.
  • Chandra added a new event for when a Provisiond node scan starts.
  • Dustin did more work on replacing the remote poller.
  • Chandra and Jesse made some enhancements to flow aggregation.
  • Marcel worked on updating the JDBCQueryMonitor and SystemExecuteMonitor documentation.
  • Chandra did more work on adding OpenTracing support to Provisiond.
  • Jesse added a collector for applications instrumented with Prometheus.
  • Patrick worked on some updates to make sure only one interface is marked SNMP primary.
  • Pierre worked on additional confd options for the Minion.
  • Patrick updated node enrichment to work on OIA topology providers.
  • Patrick added tooltip support to the legacy topology provider.
  • Patrick worked on an ALEC provider for the new graph API.
Web, ReST, UI, and Helm
  • Chandra added a smoke test for the password change UI.
  • Chandra did work on restoring the universal search bar.
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jesse White
  • Marcel Fuhrmann
  • Matthew Brooks
  • Patrick Schweizer
  • Pierre Bouffard
  • Ronny Trommer

Calendar of Events

May Releases - May 5th, 2020

The next OpenNMS release day is May 5th, 2020. Currently we expect point/bug fix releases of Meridian 2019 and Horizon 26.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • HELM-238: Circuit breaker exception in Flow Deep Dive Dashboard
  • HELM-239: Too many buckets exception in Flow Deep Dive Dashboard
  • NMS-12374: Add opentracing support for Provisiond
  • NMS-12577: Prometheus collector
  • NMS-12645: Add selenium test for password change
  • NMS-12646: Re-enable Central Search functionality
  • NMS-12647: Update smoketests to support various Kafka compression codecs
  • NMS-12648: Bump ES version used in Smoke Tests
  • NMS-12653: Remove remote-poller runtimes from the build
  • NMS-12654: Update remote-poller model to link service to locations instead of individual RPs
  • NMS-12656: Aggregate flow metrics w/ stream processing
  • NMS-12658: Provisiond: Add NodeScanStarted event for scheduled scans
  • NMS-12661: Add more context to Response Time resources (Kafka Producer)
  • NMS-12663: Flow aggregation - alternate indices based on duration of time range filter
  • NMS-12671: BMP parse error for path attribute MP_UNREACH_NLRI
  • NMS-12673: Authenticated RCE vulnerability via ActiveMQ Minion payload deserialization
  • NMS-12674: Enable node enrichment for Topology providers comming from the Integration Api

by RangerRick at May 04, 2020 06:44 PM

April 29, 2020

OpenNMS Horizon 26.0.1 (Luchador) Released

Release 26.0.1 is the second release in the Horizon 26 series.

It is an off-schedule release to fix a vulnerability in ActiveMQ and the Minion. Thanks to Florian Hauser of Code White for catching this one.

The codename for 26.0.1 is Luchador.

Bug

  • Authenticated RCE vulnerability via ActiveMQ Minion payload deserialization (Issue NMS-12673)

by RangerRick at April 29, 2020 06:30 PM

OpenNMS Meridian 2019.1.6 (Europa) Released

Release 2019.1.6 is the seventh release in the Meridian 2019 series.

It is an off-schedule release to fix a vulnerability in ActiveMQ and the Minion. Thanks to Florian Hauser of Code White for catching this one.

The codename for 2019.1.6 is Europa.

Bug

  • Authenticated RCE vulnerability via ActiveMQ Minion payload deserialization (Issue NMS-12673)

by RangerRick at April 29, 2020 06:00 PM

OpenNMS Meridian 2018.1.18 (Wildfire) Released

Release 2018.1.18 is an off-schedule release to fix a vulnerability in ActiveMQ and the Minion. Thanks to Florian Hauser of Code White for catching this one.

The codename for 2018.1.18 is Wildfire.

Bug

  • Authenticated RCE vulnerability via ActiveMQ Minion payload deserialization (Issue NMS-12673)

by RangerRick at April 29, 2020 05:30 PM

April 21, 2020

New in OpenNMS: BGP Monitoring Protocol (BMP) Functionality

With the release of version 26.0 (Balaclava), OpenNMS now offers support for the BGP Monitoring Protocol (BMP) in the Streaming Telemetry feature (telemetryd).

What Is BGP Monitoring?

The BGP Monitoring Protocol (BMP) monitors Border Gateway Protocol (BGP) sessions and provides a convenient interface to monitor BGP routing information on the routing device. The integration in OpenNMS allows you to use this information, status updates, and statistics for advanced monitoring and management, providing visibility into what BGP is doing.

OpenNMS BMP Integration

OpenNMS leverages OpenBMP, an application that was developed as a reference implementation for receiving BMP messages from network equipment and presenting the state of the BGP network to the user. (For more information on OpenBMP, check out this presentation by owner/contributor Tim Evens or the following links: OpenBMP Server Collector, OpenBMP readme.)

BGP runs on routers and high-end switches. Those routers can point to OpenNMS or an OpenNMS Minion and send BMP data to OpenNMS. To use OpenNMS BMP features, you need the OpenBMP PostgreSQL component installed and set up, and have a BMP-enabled router.

The OpenNMS BMP parser accepts BMP connections from router packets using a TCP listener. Three adapters in OpenNMS read BMP data including an inventory list of connected routers and the peers they are established with, and stats and data about what is being exchanged over those peers:

  • BMP Telemetry Adapter: handles BMP statistics received and parsed by the BMP Parser. Statistics received from the router are associated as performance data with that router. View metrics and statistics related to that peer and apply thresholding.
  • BMP Peer Status Adapter: handles BMP Peer Up and Down messages that the BMP Parser receives and parses, and converts them to OpenNMS events. Allows alarm creation based on up/down status.
  • OpenBMP Integration Adapter: integrates with an existing OpenBMP installation and handles BMP messages from the BMP Parser, and creates OpenBMP-compatible messages, which are then passed to the OpenBMP Kafka cluster.

The parser and adapters are configurable in the telemetryd-configuration.xml file. For more information on BMP functionality in OpenNMS, see the online user documentation.

Visualization

View BMP data in the OpenNMS UI by logging in and choosing Reports>Resource Graphs. Select the BMP-enabled resource for which you want to view statistics, and choose Graph All. OpenNMS displays graphs similar to the following:

Displays times when a peer withdrew an IP prefix:

Displays times when a peer withdrew an update:

You can also create dashboards in Grafana to view this OpenNMS BMP data.

Displays prefixes in violation of Resource Public Key Infrastructure (RPKI) (table and chart format), prefixes in violation of an Internet Routing Registry (IRR) (table and chart format), and the number of RPKI route origin authorizations (ROAs) and IRR entries:

Displays the number of advertised IP addresses and their IP version (in this case, IPv4), the upstream/downstream autonomous systems (ASNs), and a visualization of the originating prefix trend:

Upcoming BMP Features

This first milestone provides a high-capacity and resilient infrastructure for routing BMP messages into an SNAS/OpenBMP-compatible persistence layer.

Milestone two will incorporate all OpenBMP functionality and alarm workflows into OpenNMS workflows, the Web UI, and the Helm console. This integration will allow full support of BGP monitoring in a production environment from an holistic fault and performance monitoring system.

Our third milestone will offer a cloud-hybrid software-as-a-service (SaaS) solution where on-premise OpenNMS solutions can incorporate this feature without having to deploy the additional required infrastructure.

Sponsored Development

BMP functionality in OpenNMS is the result of a customer request through our sponsored development program. Our customers have requested and received features like SNMP version 3 support, a remote monitoring method, integration with other tools. and specific monitors for such things as SMS messaging. This collaboration between customers and open source is a great way for us to respond to customer needs while developing new product features that benefit all OpenNMS users.

by Bonnie at April 21, 2020 02:33 PM

April 20, 2020

OpenNMS On the Horizon – April 20th, 2020 – Provisiond, BMP, Documentation, Infrastructure, Response Time, OpenNMS.js, Search, and More!

It's time for OpenNMS On the Horizon!

In the last week we worked on tracing in Provisiond, BMP updates, documentation, test and build infrastructure, response time resources, OpenNMS.js, search, and other bug fixes.

Github Project Updates

Internals, APIs, and Documentation
  • Chandra did more work on adding OpenTracing support to Provisiond
  • Chandra fixed an NPE in the Kafka flow forwarder
  • Bonnie made additional documentation updates for BMP and some other core changes
  • Dustin improved handling of unknown variants in BMP and BGP data
  • I fixed a number of issues in unit/integration/smoke testing
  • Chandra improved response time resources written to Kafka to provide more info
  • I did more work on moving Bamboo tasks over to CircleCI
Web, ReST, UI, and Helm
  • I released a new version of OpenNMS.js with a fix for newer Grafana HTTP response handling
  • Chandra worked on re-adding Markus's new central search facility (including missing smoke test coverage)
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jesse White
  • Ronny Trommer

Calendar of Events

May Releases - May 5th, 2020

The next OpenNMS release day is May 5th, 2020. Currently we expect point/bug fix releases of Meridian 2019 and Horizon 26.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • HELM-232: Support Grafana 6.7.x for our Helm plugin
  • NMS-12552: BMP parser is too strict for unknown elements / types
  • NMS-12660: NPE in KafkaFlowForwarder

by RangerRick at April 20, 2020 09:00 PM

April 13, 2020

OpenNMS.js v2.0.2

Just a small bugfix release to facilitate OpenNMS Helm fixes.

Bug Fixes

  • rest: fix response type handling in grafana 6.7 (HELM-232) (874cd80)

by RangerRick at April 13, 2020 09:17 PM

OpenNMS On the Horizon – April 13th, 2020 – Documentation, BMP, Kafka, Provisiond, Bug Fixes, and More!

It's time for OpenNMS On the Horizon!

In the last week we fixed a bunch of bugs, did more BMP work, made build system improvements, made Provisiond improvements, and a bunch more.

Github Project Updates

Internals, APIs, and Documentation
  • Chandra fixed an NPE that could cause startup errors in the graph service.
  • Bonnie did more updates to the thresholding documentation and PDF dashboards.
  • Ronny fixed an issue with Docker image creation when Confd download fails.
  • I made some improvements to CircleCI cache handling for builds.
  • Christian, Dustin, and Jesse fixed some issues with BMP parsing and config loading.
  • Dustin implemented some deprecated OSPF attributes in BMP handling.
  • Sean added some attributes to the RPM specs to improve compatibility.
  • Jesse updated the OIA used by OpenNMS to 0.4.0.
  • Dustin and Bonnie wrapped up adding BMP documentation.
  • Sean worked on Kafka updates to support compression and ES updates.
  • Jesse fixed some issues with DNS lookup while BMP parsing.
  • Patrick worked on an InfluxDB plugin for the new timeseries API.
  • Patrick fixed an issue with Provisiond if multiple interfaces are (incorrectly) marked primary.
  • Chandra added tracing support to Provisiond.
  • Chandra added a nodeScanStarted event for when a node scan is scheduled during provisioning or a scheduled run.
Web, ReST, UI, and Helm
  • Patrick fixed an issue with creating monitored service through the v1/v2 ReST APIs.
Contributors

Thanks to all of the following contributors for committing changes since last OOH:

  • Benjamin Reed
  • Bonnie Robinson
  • Chandra Gorantla
  • Christian Pape
  • Dustin Frisch
  • Jesse White
  • Patrick Schweizer
  • Ronny Trommer
  • Sean Torres

April OpenNMS Meridian and Horizon Releases

OpenNMS Meridian

Last week we released updates to Meridians 2017 through 2019. These are recommended updates because they include security fixes:

OpenNMS Horizon

On top of the Meridian point releases, we also put out Horizon 26, a new major version of the OpenNMS Horizon track.

The biggest change involves adding support for processing BMP (BGP Monitoring Protocol) in Telemetryd.

If you'd like to see an overview of the major changes since Horizon 25, take a look at What’s New in OpenNMS Horizon 26. For an exhaustive list of issues that have been resolved since Horizon 25, see the change log.

Calendar of Events

May Releases - May 5th, 2020

The next OpenNMS release day is May 5th, 2020. Currently we expect to point/bug fix releases of Meridian 2019 and Horizon 26.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-10597: Limit SNMP Datacollection by Table Index
  • NMS-12532: Selecting an Icon on Topology Map breaks the map
  • NMS-12585: Evaluate stream processing for Top N calculation with flows
  • NMS-12588: Create threshold documentation
  • NMS-12599: Document how to generate PDFs from dashboards using OpenNMS
  • NMS-12633: Timeseries Plugin Influx 1.x
  • NMS-12636: Can't change password using the user self service
  • NMS-12637: GraphService is throwing Error with an NPE Karaf startup
  • NMS-12638: Telemetryd with BMP adapter throws java.util.ConcurrentModificationException
  • NMS-12639: Add required dependencies to use ZSTD inside Kafka to features.xml
  • NMS-12640: Set RPM compression type and level inside RPM Spec Files
  • NMS-12649: Error parsing label information from BGP MP_REACH_NLRI attribute

by RangerRick at April 13, 2020 04:35 PM

April 09, 2020

OpenNMS Horizon 26.0.0 (Balaclava) Released

Release 26.0.0 is the first release in the Horizon 26 series.

It contains a large number of bug fixes and new features, most notably initial support for handling the BGP Monitoring Protocol in Telemetryd.

For a high-level overview of what has changed in Horizon 26, see What’s New in OpenNMS Horizon 26.

The codename for 26.0.0 is Balaclava.

by RangerRick at April 09, 2020 08:00 PM

April 07, 2020

OpenNMS Meridian 2019.1.5 (Saturn) Released

Release 2019.1.5 is the sixth release in the Meridian 2019 series.

It fixes a few more security issues, as well as a number of other bugs and a couple of enhancements. Hat tip to Johannes Moritz for the security report.

The codename for 2019.1.5 is Saturn.

Bug
  • SNMP Remove from definitions fails for definitions with profile label (Issue NMS-12413)
  • persisted defaultCalendarReport database reports are broken (Issue NMS-12438)
  • Security issue disclosures, 31 Jan 2020 (Issue NMS-12513)
  • Selecting an Icon on Topology Map breaks the map (Issue NMS-12532)
  • Description: Cannot create monitored-service with JSON via ReST (Issue NMS-12625)
  • Confd download fails silently on Docker install (Issue NMS-12642)
Enhancement
  • Event documentation is missing tokens (Issue NMS-12228)
  • Splitting Docker documentation in Horizon, Minion and Sentinel (Issue NMS-12529)
  • Improve OIA performance when mapping alarms (Issue NMS-12581)
  • Events not balanced across partitions when using opennms-kafka-producer (Issue NMS-12616)

by RangerRick at April 07, 2020 08:06 PM

OpenNMS Meridian 2018.1.17 (Pandemic) Released

Release 2018.1.17 is a small update to 2018.1.16 that fixes another security issue that affects most current OpenNMS releases. Hat tip to Johannes Moritz for reporting this.

The codename for 2018.1.17 is Pandemic.

Bug
  • Security issue disclosures, 31 Jan 2020 (Issue NMS-12513)
  • Drools working memory facts are not restored properly on engine reload (Issue NMS-12586)
  • Confd download fails silently on Docker install (Issue NMS-12642)
Story
  • Backport CircleCI pipeline to foundation-2018 (Issue NMS-12476)

by RangerRick at April 07, 2020 07:20 PM

OpenNMS Meridian 2017.1.22 (meridianu(s) Budense) Released

Release 2017.1.22 is a small update to 2017.1.21 that fixes another security issue that affects most current OpenNMS releases. Hat tip to Johannes Moritz for reporting this.

The codename for 2017.1.22 is meridianu(s) Budense.

Bug

  • Security issue disclosures, 31 Jan 2020 (Issue NMS-12513)
  • Confd download fails silently on Docker install (Issue NMS-12642)

Enhancement

  • Backport CircleCI pipeline to foundation-2017 (Issue NMS-12603)

by RangerRick at April 07, 2020 06:47 PM

April 06, 2020

OpenNMS On the Horizon – April 6th, 2020 – BMP, Kafka, Documentation, CircleCI, Provisiond, Bug Fixes, and More!

It's time for OpenNMS On the Horizon!

In the last week we did more CircleCI infrastructure work, continued to work on BMP features, did more documentation updates, and did a bunch of small feature work and bugfixing.

Github Project Updates

Internals, APIs, and Documentation
  • Dustin, Christian, and Jesse continued their work on improvements to BMP integration.
  • Jesse fixed the minion container to not enable JMS when Kafka is enabled.
  • Bonnie continued her work on improving thresholding documentation.
  • Chandra did more work on the conversion to using protobuf instead of bson for flow payloads.
  • I implemented foundation and release branch merging in CircleCI and turned them off in Bamboo.
  • Jeff disabled logging RTC events by default in Horizon 26 and up.
  • Ronny updated the Horizon and Sentinel CircleCI config to push RC branches.
  • Ronny did more work on updating Helm documentation generation.
  • Ronny updated Minion in smoke tests to run as non-root.
  • Chandra worked on adding tracing support to Provisiond.
  • Patrick worked on a few web UI fixes.
  • Sean worked on updating RPM builds for compatibility.
  • Sean worked on making it possible to ZSTD in Kafka.
Web, ReST, UI, and Helm
  • I fixed documentation generation in OpenNMS.js.
  • I fixed a bug in d3 event handling in the Topology UI.

Calendar of Events

April Releases - April 7th, 2020

The next OpenNMS release day is April 7th, 2020.

Unless we run into major issues, we're hoping to release Horizon 26 in April, which, among other things, includes support for BMP telemetry collection.

There will also be updates to all active Meridian releases.

Until Next Week…

If there’s anything you’d like me to talk about in a future OOH, or you just have a comment or criticism you’d like to share, don’t hesitate to say hi.

- Ben

Resolved Issues Since Last OOH

  • NMS-8032: Describe Alarm status behavior in user documentation
  • NMS-8546: Event to Notification matching is not documented
  • NMS-9754: RTC subscription events should not be persisted to DB
  • NMS-12426: Add BMP config example and documentation
  • NMS-12513: Security issue disclosures, 31 Jan 2020
  • NMS-12521: Use protobuf instead of bson for encoding/decoding Netflow payloads
  • NMS-12559: Improve parsing of BGP extended communities attribute
  • NMS-12560: Populate path id and labels attributes in unicast prefix messages (OpenBMP integration)
  • NMS-12569: Async DNS resolution for Hostnames in BMP
  • NMS-12617: XSS security issues
  • NMS-12619: sentinel-coordination-zookeeper doesn't start due to missing dependency
  • NMS-12625: Description: Cannot create monitored-service with JSON via ReST
  • NMS-12628: auto-merge foundation branches in CircleCI
  • NMS-12630: Push Minion OCI to DockerHub for release branches
  • NMS-12631: Minion confd template should disable JMS when using Kafka
  • NMS-12635: Restore CAP_NET_RAW capabilities in Minion when running as non-root
  • NMS-12641: Support for more extended community types in BMP
  • NMS-12642: Confd download fails silently on Docker install
  • NMS-12643: Error parsing MP_UNREACH_NLRI attribute
  • NMS-12644: BMP Parser Bulkhead Config does not work

by RangerRick at April 06, 2020 09:43 PM

April 03, 2020

Happy 20th Birthday, OpenNMS!

Earlier this week, OpenNMS turned 20 years old. Although the project began in the summer of 1999, March 30, 2000, marks the day that the OpenNMS Project was registered on Sourceforge and the first time any OpenNMS code was made public. David Hustace, CEO of OpenNMS and Tarus Balog, the company’s COO, share their memories of the past and thoughts about the future.

A Radical Idea in 2000

“The term "open source" was created in 1998 on the heels of the release of the Netscape browser source code, and the OSI [Open Source Initiative] was founded,” says David. “I can't provide an insider perspective on what it was like when OpenNMS first started, but I can tell you that, from an outsider's perspective, it was an extremely radical idea.”

“In 2000, the dot com bubble was subsiding but open source was never going to be considered something to be taken seriously,” he adds. “There was an onslaught of fear, uncertainty and doubt (FUD) coming from a lot of the software industry at the time.”

Network Management Has Changed a Lot

“Back then, most enterprise-level monitoring solutions required expensive hardware from IBM, HP or Sun and the scope was very limited, mainly focused on connectivity and performance network devices such as switches and routers,” says Tarus. “Some vendors were just starting to support Windows, and Linux support was non-existent. Now network monitoring solutions flourish at all levels and on all operating systems.”

David points to virtualization, IoT, and the Cloud, as three of the most significant changes in network management, because of the complexity they add. “Virtualization means monitored entities are now completely abstracted and almost always virtual implementations of their former physical selves; yet they present themselves identically to a monitoring system,” he says. “Moving monitoring applications to the Cloud introduces many application security and networking communications hurdles to crossover, as network and computer technologies still exist in data centers and corporate offices. Monitoring applications that are running in the Cloud require access to those local, on-premise networks to reliably monitor faults and performance; especially when the Cloud applications often require traversing the public Internet. And of course, IoT has changed monitoring with respect to scale. The number of devices on the network needing to be monitored has changed by multiple orders of magnitude due to the compression of technology into almost every aspect of our lives.”

Open Source: from Fringe to Mainstream

“Open source has moved from a fringe idea from a bunch of free software advocates to mainstream acceptance,” says Tarus. “We used to have much more of an ‘evangelical sale’ of OpenNMS in an organization because no one understood it well, or they were frightened by some of the FUD spread by proprietary software vendors. We no longer have that problem, as many large companies have embraced open source from the operating system on up to the software they integrate into their solutions.”

Noteworthy OpenNMS Achievements

“For me it is the fact that we are still around,” says Tarus. “OpenNMS was started by a company called Oculan. Before they went out of business in 2004, they decided to no longer support OpenNMS in May of 2002. While I might have had the option to remain at Oculan, I saw the potential of an open source network monitoring platform that could scale and decided to take a leap of faith and keep the project alive. I was lucky in that I met people like David and was able to convince them of the potential in OpenNMS, enough so that they would quit well-paying jobs to pursue making OpenNMS awesome.”

David laughs. “Twenty years! I thought of this just today … there are potentially users of OpenNMS that are younger than the project. OMG,” he says. “I think it’s significant that we still have our first customer as a customer.”

If You Could Go Back in Time

Both men reflected on what would surprise that younger self if they could go back in time and talk to him.

“That it is worth it,” says Tarus. “Trust me, in the years I’ve worked with OpenNMS there were a lot of times I didn’t think the project was going to make it. It has been the best job I’ve ever had but also the most work.”

“I would be most surprised that this project has these incredibly smart, talented, and wonderful human beings working together and producing an application that is used in production by tens of thousands of very large corporations and most of the biggest, with a combined valuation of trillions of US dollars,” says David.

Where Will OpenNMS be in 20 Years?

In 2000, no one could predict how networking and open source would change and develop. But David and Tarus still have visions for where they want OpenNMS to be 20 years from now. For David, winning the National Medal of Technology and Innovation would be a great honour and achievement.

Tarus’ goal is for OpenNMS to be “the de facto monitoring platform for carriers and large enterprises,” he says.

Personal Stories

Beyond OpenNMS’s success and staying power, it’s the people that have had the greatest impact.

Tarus describes how the company was able to hire one of its contributors, and bring him to the US on an H1B visa. “That eventually turned into a permanent resident card for him and his wife, and his future is now only limited by his imagination,” he says.” If OpenNMS went away tomorrow it would be all worth it for just that one thing, and there are lots of other examples that make all the effort worthwhile.”

Adds David, “OpenNMS and open source has enriched my life by introducing me to, and allowing me to, collaborate and experience life with the most wonderful people in the world.”

by Bonnie at April 03, 2020 05:26 PM

March 30, 2020

It Was Twenty Years Ago Today …

On March 30th, 2000, the OpenNMS Project was registered on Sourceforge. While the project actually started sometime in the summer of 1999, this was the first time OpenNMS code had been made public so we’ve always treated this day as the birth date of the OpenNMS project.

Wow.

OpenNMS Entry on Sourceforge

Now I wasn’t around back then. I didn’t join the project until September of 2001. When I took over the project in May of 2002 I didn’t really think I could keep it alive for twenty years.

Seriously. I wasn’t then nor am I now a Java programmer. I just had a feeling that there was something of value in OpenNMS, something worth saving, and I was willing to give it a shot. Now OpenNMS is considered indispensable at some of the world’s largest companies, and we are undergoing a period of explosive growth and change that should cement the future of OpenNMS for another twenty years.

What really kept OpenNMS alive was its community. In the beginning, when I was working from home using a slow satellite connection, OpenNMS was kept alive by people on the IRC channel, people like DJ and Mike who are still involved in the project today. A year or so later I was able to convince my business partner and good friend David to join me, and together we recruited a real Java programmer in Matt. Matt is no longer involved in the project (people leaving your project is one of the hardest things to get used to in open source) but his contributions in those early days were important. Several years after that we were joined by Ben and Jeff, who are still with us today, and through slow and steady steps the company grew alongside the project. They were followed by even more amazing people that make up the team today (I really want to name every single one of them but I’m afraid I’ll miss one and they’ll be rightfully upset).

I can’t really downplay enough my lack of responsibility for the success of OpenNMS. My only talent is getting amazing people to work with me, and then I just try to remove any obstacles that get in their way. I get some recognition as “The Mouth of OpenNMS” but most of the time I just stand on the shoulders of giants and enjoy the view.

by Tarus at March 30, 2020 09:03 PM