Writing plugins

Writing plugins for monometric.io is really easy. We have created a flexible and powerful plugin system, that allows you to write plugins in any language you like. You will literally be up and running in seconds!

Example #1: Getting started

Writing a bash plugin that reports the number of connections we have from or to MySQL:

#!/bin/bash
echo mysql.connections: `netstat -tn | grep ESTABLISHED | grep :3306 | wc -l`

Save this plugin as /etc/mm-agent/plugins/mysql_connections.sh, and make it executable by running

[root@example plugins]# chmod +x /etc/mm-agent/plugins/mysql_connections.sh

Then we test what the output will look like

[root@example plugins]# ./mysql_connections.sh
mysql.connections: 23
[root@example plugins]#

And that's it.

The agent will automatically detect and run our new plugin, no restart required.

We now have a fully funtional, really simple plugin that graphs the number of connections with a source or destionation port of 3306, the MySQL default port. We can graph this value in the monometric.io webinterface, and setup alerts that will notify us if the number drops below three.

If you need to disable the plugin, simply delete it, or run:

[root@example plugins]# chmod -x /etc/mm-agent/plugins/mysql_connections.sh

NOTE: The script will run as an unprivileged user per default, so make sure you actually have access to run the commands in your plugin. If you need to run your plugins as root, simply put them in the /etc/mm-agent/privileged_plugins folder.

Example #2: Another example

Writing a bash plugin to check the ping time to google.com:

#!/bin/bash
echo pingplugin.ms: `ping -c 1 google.com | grep ^rtt |cut -d/ -f 5`

Save this plugin as /etc/mm-agent/plugins/pingplugin.sh, and make it executable by running

[root@example plugins]# chmod +x /etc/mm-agent/plugins/pingplugin.sh

The plugins output will be:

[root@example plugins]# ./pingplugin.sh
pingplugin.ms: 34.231
[root@example plugins]#

The filename doesn't really matter, but the file needs to be executable and readable by the mm-agent user.

Example #3: Running with root privileges and adjusting execution interval

Writing a python script, that scans our access log for unauthorized login attempts:

#!/usr/bin/python
num_lines = 0
unknown_users = 0
fh = open('/var/log/secure')
if fh:
    while True:
        line = fh.readline()
        if not line:
            break
        if line.find('Invalid user ') != -1:
            unknown_users += 1
        num_lines +=1
print "securelog.lines:", num_lines
print "securelog.unknown_users:", unknown_users

In this case, it is important that we put it in /etc/mm-agent/privileged_plugins, as only root can read /var/log/secure on most systems.

We verify the output:

[root@example privileged_plugins]# ./scanlog.py
securelog.lines: 2650
securelog.unknown_users: 636
[root@example privileged_plugins]#

The output format is very simple. A line starts with a metric name, followed by colon, and then a value, which can be any string or number. For the key name, only a-z, 0-9 and dots are allowed. A plugin can have several key-value fields as output, separated by a line.

In this example, we scan through our /var/log/secure logfile, and counts lines invalid users. The default plugin execution interval is 10 seconds. As this information isn't too volatile, we can get away with only doing it once every minute, to save resources on our server. To do this, we create a config for our plugin, called scanlog.py.conf:

[plugin]
interval = 60

This way, we instruct the agent to run it at most every 60 seconds. Even if the plugin is very slow, i.e. it takes 70 seconds to finish, the agent will never run more than one instance at once.

Example #4: Working with rates and counters

The python script we just wrote, also reports the number of lines in our logfile. This number isn't very interesting in it self, it is just a counter. What would be more interesting, would be to see the rate of new lines, or rate of invalid login attempts in our logfile. The agent has built in support for this. The only thing we need to do is prefix the key name with _counter.

#!/usr/bin/python
num_lines = 0
unknown_users = 0
fh = open('/var/log/secure')
if fh:
    while True:
        line = fh.readline()
        if not line:
            break
        if line.find('Invalid user ') != -1:
            unknown_users += 1
        num_lines +=1
print "_counter.securelog.lines:", num_lines
print "_counter.securelog.unknown_users:", unknown_users

New output:

[root@root privileged_lugins]# ./scanlog.py
_counter.securelog.lines: 2849
_counter.securelog.unknown_users: 699
[root@root privileged_lugins]#

Now, the agent will automatically detect this, and we can graph this values as value per second. If you would like, you can also multiply the number with 60, to get value per minute by making these small changes to the script:

print "_counter.securelog.lines.per_min:", num_lines * 60
print "_counter.securelog.unknown_users.per_min:", unknown_users * 60

Example #5: Environment variables

If your process needs environment variables, these can be specified in the .conf-file, like the plugin interval. One usecase for this, is code reuse. Let's say we want to ping several hosts, but use the same code.

We go back back to example #2, but modify the example slightly.

#!/bin/bash
echo pingplugin.ms.$MY_PLUGIN_METRIC_NAME: `ping -c 1 $MY_PLUGIN_HOST | grep ^rtt |cut -d/ -f 5`

We save this file as pingplugin.sh. We create a config file, pingplugin.sh.conf.

[env]
MY_PLUGIN_METRIC_NAME=google
MY_PLUGIN_HOST=google.com

The variables in pingplugin.sh.conf will be injected in the plugin environment. Testing the plugin can be done like this (or just export the environment variables):

[root@example DO /etc/mm-agent/plugins]# MY_PLUGIN_METRIC_NAME=google MY_PLUGIN_HOST=google.com ./pingplugin.sh
pingplugin.ms.google: 0.285
[root@example DO /etc/mm-agent/plugins]#

Community

Ultimately, only your imagination limit what you can monitor from custom plugins. Make sure to identify what is most important for you, and setup alerts. If you are looking for insipiration, head over to our community plugins page (to be announced). Maybe you can find exactly what you are looking for already.