Need to add phone and SMS alerting to ClusterControl? ClusterControl 1.2.8 introduces support for PagerDuty, an alerting service for Ops teams to schedule on-calls and add phone and SMS notifications to IT tools. By integrating PagerDuty with ClusterControl, you can start receiving phone, SMS and email notifications for all important database events as monitored by ClusterControl. Alerts go directly to the right person who can solve the issue.
This integration is possible thanks to a new plugin interface, that takes ClusterControl alarms in JSON format and outputs to an external system via plugins. Plugins can be either scripts or executable binaries.
We have built a few example plugins utilizing this plugin interface, available from our Github repository:
- pagerduty.py: This plugin forwards the alarm raise/close events to the PagerDuty system
- syslog.py: This plugin writes the new alarms instantly to the syslog
Plug-in configuration options
Let’s have a quick look at how the plugin works, but feel free to go directly to the PagerDuty setup instructions.
A plugins directory can be set through the CMON configuration file (config file entry is plugin_dir), or in the cmon_configuration table found in CMON DB (PLUGIN_DIR key).
When no value is set, the /var/cmon/plugins path will be used. The controller tries to execute the 'executable' scripts/binaries from the directory, while the non-executable files are skipped.
How does the plugin execute?
Whenever an alarm is raised, CMON feeds the alarm event to its standard input (stdin) and all executable scripts or binaries (plugins) under plugin_dir path will be executed. A JSON message will be written to those standard inputs. The expected JSON message syntax is as follows:
{// currently only the "alarm" is supported"type":"alarm",// whether it is a new alarm, or an update for an old alarm or an alarm removal"action":"new|update|remove",// a JSON map which contains the alarm details"alarm":{},// the hostname of the cmon controller (could be used to determine the web-ui url)"cmon_hostname":"ip-address/or-host-name"}