[icinga-users] Icinga2 pnp4nagios Performance Data from applied Service

Yuval Freund yuv.freu at gmail.com
Sun Aug 10 10:44:43 CEST 2014


Ok, so the problem doesn't seem to have anything to do with the service
being "applied" after all - for some reason, the performance data coming
from the ping command is being converted from ms to seconds, and instead of
returning 0s it just returns s.

Changed pnp4nagios logging to debug, and got the following for all ping
services:

2014-08-10 10:27:45 [30398] [2] No pattern match in function _parse(s;3;5;0
)

Not sure which process along the way is doing that yet (npcd?), but I'll
try to find out and resolve it tomorrow. If anyone has any ideas or
information, they'll be more than welcome.

@Michael - thanks for your help and guidance. I guess I should have tried
to apply some more services before assuming that was the source of the
problem. I think the Icinga2 documentation is pretty good for such a young
system, but improving it is an ongoing process, and I believe the ITL part
could use a little work (though it did help!).


On Thu, Aug 7, 2014 at 3:57 PM, Yuval Freund <yuv.freu at gmail.com> wrote:

> - Are you sure that the apply rules really generate a valid service
> object which then gets checked? Ignore the performance data part for now.
>
> If I am to trust the interface:
>
>
> Current Status:
>   OK
>  (for 13d 0h 54m 8s)Status Information:PING OK - Packet loss = 0%, RTA =
> 0.04 ms Performance Data:pl=0%;5;15;0
> rta=3.5000000000000004e-05s;0.1;0.2;0
> - That's called the ITL provided plugin check commands, and we've added a complete
> section for it to the documentation.
>
> Alright, thanks for pointing me in the right direction - found the command
> definition in /usr/share/icinga2/include/command-plugins.conf (hope this
> helps any googlers out there with similar setups ;) ).
>
> nagios at SERVERNAME:/usr/lib/nagios/plugins$ ./check_ping -H 127.0.0.1 -w
> 100,5% -c 200,15% -4
> PING OK - Packet loss = 0%, RTA = 0.04
> ms|rta=0.035000ms;100.000000;200.000000;0.000000 pl=0%;5;15;0
>
> here's some more info regarding the nagios-plugins package I'm using:
>
> Package: nagios-plugins
> State: installed
> Automatically installed: no
> Version: 1.5-3ubuntu1
> Priority: extra
> Section: net
> Maintainer: Ubuntu Developers <ubuntu-devel-discuss at lists.ubuntu.com>
>
>
> - I have plenty of ping4 and ping6 services being executed, generating valid
> performance data. I only have PNP disabled in my demo setups, I prefer
> Graphite in-memory.
>
> I should point out I tried using Graphite in a different test setup
> (played around with the "official" vagrant machines), and it worked
> flawlessly for all services, even the applied ones. But Graphite also uses
> a different connector in "features-enabled" (graphite.conf instead
> of perfdata.conf). Not sure if it makes a difference or not.
>
>
>
> On Wed, Aug 6, 2014 at 8:36 PM, Michael Friedrich <
> michael.friedrich at netways.de> wrote:
>
>> On 06.08.2014 13:48, Yuval Freund wrote:
>>
>>> - You said something about a services.conf where services exist which
>>> do not generate any perfdata and therefore rrd files.
>>>
>>> services.conf contains the applied service I pasted in my last email,
>>> and ping6, which I also don't seem to get an rrd-file from.
>>>
>>
>> Are you sure that the apply rules really generate a valid service object
>> which then gets checked? Ignore the performance data part for now.
>>
>>
>>
>>  The other services are defined directly under the host, for example -
>>> /etc/icinga2/conf.d/hosts/localhost/users.conf :
>>>
>>>
>>> object Service "users" {
>>> import "generic-service"
>>>
>>> host_name = "localhost"
>>> check_command = "users"
>>> vars.sla = "24x7"
>>>
>>> action_url = "/pnp4nagios/graph?host=$HOSTNAME$&srv=$SERVICEDESC$'
>>> class='tips' rel='/pnp4nagios/popup?host=$HOSTNAME$&srv=$SERVICEDESC$"
>>> }
>>>
>>
>> That's a service object directly bound to one host, namely 'localhost'.
>> Given the check_command attribute 'users' provided by the itl plugin
>> check commands (check the docs for details on parameters) we can assume
>> that it calls the 'check_users' plugin from the Monitoring Plugins
>> project which generates performance data by default.
>>
>>
>>
>>>
>>> The main difference I can see between services that I do get rrd-files
>>> from, and services I do not get rrd-files from, is that the ones which
>>> do not generate any files are applied per rule using "apply Service"
>>> from the main services.conf file, and the ones that do work are
>>> defined under the host per "object Service". I hope that makes my
>>> issue a bit clearer.
>>>
>>
>> You are still mixing the different possible causes. Services do not
>> generate rrd files, that's being done by a perfdata processing tool such
>> as npcd (which ignores invalid performance data).
>>
>> Even if I repeat myself, but please follow these steps:
>>
>>
>>
>>> It doesn't make sense to debug three different locations for "no graph
>>> is shown"
>>>
>>> * Verify which command is being executed given the configuration objects
>>> * Validate the check plugin manually providing the correct performance
>>> data
>>>
>>> Only if these points are done, step to npcd/process perfdata. And if
>>> their logs don't tell anything, dive into rrdtool.
>>>
>>
>>
>>
>>
>>>
>>> - It's still a sample config, and shall be revised/modified by your
>>>
>>> own strategy
>>>
>>> I fully intend to do that, once I get this issue out of the way. It's
>>> easier to debug with configs which are easily replicated, as anyone
>>> can just grab a copy, or create a vagrant instance with those configs.
>>> BTW - your link returned a "page not found" :(
>>>
>>
>> That's a framework session problem of the Icinga Web 2 doc viewer which
>> we are using in an early version. The next milestone release will
>> hopefully fix it and we may upgrade docs.icinga.org.
>>
>> In your case, that means, opening the link once again in another tab,
>> now having the session cookie.
>>
>> Details: https://dev.icinga.org/issues/6730
>>
>>
>>
>>
>>>
>>> - Run the plugin manually as Icinga 2 would execute it, and post the
>>> entire output.
>>>
>>> Sure - where does icinga2 keep the internal command files, such as
>>> ping4? I haven't found any documantation on that, and there's no path
>>> under /etc/icinga2/features-enabled/command.conf . There's no mention
>>> of ping4 in any other config file.
>>>
>>
>> That's called the ITL provided plugin check commands, and we've added a
>> complete section for it to the documentation.
>>
>> http://docs.icinga.org/icinga2/latest/doc/module/
>> icinga2/toc#!/icinga2/latest/doc/module/icinga2/chapter/
>> configuring-icinga2#icinga-template-library
>> http://docs.icinga.org/icinga2/latest/doc/module/
>> icinga2/toc#!/icinga2/latest/doc/module/icinga2/chapter/
>> configuring-icinga2#plugin-check-commands
>>
>> The icinga 2 feature called 'command' has nothing to do with that -
>> that's just the external command pipe. Find an explaination about its
>> possibilities in the 'monitoring basics' section too.
>>
>> http://docs.icinga.org/icinga2/latest/doc/module/
>> icinga2/toc#!/icinga2/latest/doc/module/icinga2/chapter/
>> monitoring-basics#external-commands
>>
>>
>>
>>>
>>> In addition to me debugging this issue, it would be nice to get some
>>> feedback from other users - Has anyone successfully applied and got
>>> PNP data from an "applied" service in icinga2 ?
>>>
>>
>> I have plenty of ping4 and ping6 services being executed, generating
>> valid performance data. I only have PNP disabled in my demo setups, I
>> prefer Graphite in-memory.
>>
>>
>> I'd be curious on your step-by-step output requested above.
>>
>>
>>
>>>
>>> On Tue, Aug 5, 2014 at 7:07 PM, Michael Friedrich
>>> <michael.friedrich at netways.de <mailto:michael.friedrich at netways.de>>
>>>
>>> wrote:
>>>
>>>     Am 05.08.2014 18:02, schrieb Yuval Freund:
>>>
>>>>     Hello and thanks for your reply,
>>>>
>>>>     - Can we see the exact configuration?
>>>>
>>>>     Sure, which configuration would you require?
>>>>     Icinga/PNP/rrdcached/.. ? Which exact files?
>>>>     As I stated in my original post, I used the supplied guide in
>>>>     order to get PNP up and running, so all configurations are
>>>>     identical to the ones there.
>>>>
>>>
>>>     You said something about a services.conf where services exist
>>>     which do not generate any perfdata and therefore rrd files.
>>>
>>>
>>>     "pnp only recieves performance data from services which are
>>>     directly configured under the host, but not for services which
>>>     were applied from services.conf ."
>>>
>>>     I was curious how they would look like.
>>>
>>>
>>>
>>>>     Here's the service configuration, which was included in the .deb
>>>>     package from the repo (the only thing added is action_url):
>>>>
>>>
>>>     It's still a sample config, and shall be revised/modified by your
>>>     own strategy. Find more on best practices on the documentation on
>>>     that:
>>>     http://docs.icinga.org/icinga2/latest/doc/module/
>>> icinga2/toc#!/icinga2/latest/doc/module/icinga2/chapter/
>>> monitoring-basics#configuration-best-practice
>>>      <http://docs.icinga.org/icinga2/latest/doc/module/
>>> icinga2/toc#%21/icinga2/latest/doc/module/icinga2/
>>> chapter/monitoring-basics%23configuration-best-practice>
>>>
>>>
>>>
>>>
>>>
>>>>
>>>>     apply Service "ping4" {
>>>>     import "generic-service"
>>>>
>>>>     check_command = "ping4"
>>>>     vars.sla = "24x7"
>>>>
>>>>     assign where "linux-servers" in host.groups
>>>>     assign where "windows-servers" in host.groups
>>>>     ignore where host.address == ""
>>>>
>>>>     action_url =
>>>>     "/pnp4nagios/graph?host=$HOSTNAME$&srv=$SERVICEDESC$'
>>>>     class='tips'
>>>>     rel='/pnp4nagios/popup?host=$HOSTNAME$&srv=$SERVICEDESC$"
>>>>     }
>>>>
>>>>
>>>>     And here's a listing of the perfdata/localhost folder, so you
>>>>     could see other services are creating and populating rrd-files:
>>>>
>>>>     root at SERVERNAME:/var/lib/pnp4nagios/perfdata/localhost# ll
>>>>     total 22188
>>>>     drwxrwxr-x 2 nagios nagios 4096 Aug 5 14:17 ./
>>>>     drwxr-xr-x 4 nagios nagios 4096 Jul 25 15:32 ../
>>>>     -rw-rw-r-- 1 nagios nagios 768224 Aug 5 14:00 apt.rrd
>>>>     -rw-rw-r-- 1 nagios nagios 2530 Aug 5 14:16 apt.xml
>>>>     -rw-rw-r-- 1 nagios nagios 3067856 Aug 5 14:14 disk.rrd
>>>>     -rw-rw-r-- 1 nagios nagios 6915 Aug 5 14:16 disk.xml
>>>>     -rw-rw-r-- 1 nagios nagios 768224 Jul 29 00:35 _HOST_.rrd
>>>>     -rw-rw-r-- 1 nagios nagios 2455 Jul 29 00:35 _HOST_.xml
>>>>     -rw-rw-r-- 1 nagios nagios 768224 Aug 5 14:09 http.rrd
>>>>     -rw-rw-r-- 1 nagios nagios 2512 Aug 5 14:16 http.xml
>>>>     -rw-rw-r-- 1 nagios nagios 14566016 Aug 5 14:03 icinga.rrd
>>>>     -rw-rw-r-- 1 nagios nagios 27147 Aug 5 14:16 icinga.xml
>>>>     -rw-rw-r-- 1 nagios nagios 1151496 Aug 5 14:11 load.rrd
>>>>     -rw-rw-r-- 1 nagios nagios 3176 Aug 5 14:17 load.xml
>>>>     -rw-rw-r-- 1 nagios nagios 384952 Aug 5 13:48 procs.rrd
>>>>     -rw-rw-r-- 1 nagios nagios 1874 Aug 5 14:17 procs.xml
>>>>     -rw-rw-r-- 1 nagios nagios 384952 Aug 5 13:46 ssh.rrd
>>>>     -rw-rw-r-- 1 nagios nagios 1882 Aug 5 14:17 ssh.xml
>>>>     -rw-rw-r-- 1 nagios nagios 384952 Aug 5 14:13 swap.rrd
>>>>     -rw-rw-r-- 1 nagios nagios 1956 Aug 5 14:16 swap.xml
>>>>     -rw-rw-r-- 1 nagios nagios 384952 Aug 5 13:54 users.rrd
>>>>     -rw-rw-r-- 1 nagios nagios 1862 Aug 5 14:16 users.xml
>>>>
>>>
>>>     It doesn't make sense to debug three different locations for "no
>>>     graph is shown"
>>>
>>>     * Verify which command is being executed given the configuration
>>>     objects
>>>     * Validate the check plugin manually providing the correct
>>>     performance data
>>>
>>>     Only if these points are done, step to npcd/process perfdata. And
>>>     if their logs don't tell anything, dive into rrdtool.
>>>
>>>
>>>>
>>>>     - Verify that these check commands and plugins provide valid
>>>>     performance data.
>>>>
>>>>     "..even though I can see the performance data under "Service
>>>>     State Information" ". Here's what I see under the service ping4
>>>>     as an example - Performance Data:pl=0%;5;15;0
>>>>     rta=3.1999999999999999e-05s;0.1;0.2;0
>>>>
>>>
>>>     Run the plugin manually as Icinga 2 would execute it, and post the
>>>     entire output.
>>>
>>>
>>>>
>>>>
>>>>     - Hard to tell with that little details you provided.
>>>>
>>>>     "Please let me know if you require any additional details from my
>>>>     end." :)
>>>>
>>>
>>>     I'd remove the wiki guide, and move it to the documentation. I'm
>>>     not very keen on a split mode between the wiki and the docs, as
>>>     happened with 1.x. If you have anything to add to the docs, please
>>>     send a patch to the issue - https://dev.icinga.org/issues/6858
>>>
>>>     http://docs.icinga.org/icinga2/latest/doc/module/
>>> icinga2/toc#!/icinga2/latest/doc/module/icinga2/chapter/
>>> addons-plugins#addons-graphing-pnp
>>>      <http://docs.icinga.org/icinga2/latest/doc/module/
>>> icinga2/toc#%21/icinga2/latest/doc/module/icinga2/
>>> chapter/addons-plugins%23addons-graphing-pnp>
>>>
>>>     regards,
>>>     Michael
>>>
>>>
>>>
>>>
>>>>
>>>>     On Tue, Aug 5, 2014 at 5:44 PM, Michael Friedrich
>>>>     <michael.friedrich at netways.de
>>>>     <mailto:michael.friedrich at netways.de>> wrote:
>>>>
>>>>         Am 05.08.2014 17:17, schrieb Yuval Freund:
>>>>
>>>>>         Hello,
>>>>>
>>>>>         I set up the latest version (2.0.1) of Icinga2 on an Ubuntu
>>>>>         14.04 Machine with ClassicUI and pnp4nagios using the
>>>>>         official repository. Everything seems to be working,
>>>>>         including my integration with the ClassicUI, accept for one
>>>>>         issue - pnp only recieves performance data from services
>>>>>         which are directly configured under the host, but not for
>>>>>         services which were applied from services.conf .
>>>>>
>>>>
>>>>         Can we see the exact configuration?
>>>>
>>>>
>>>>
>>>>>         Example - With the supplied test configs, I can see graphs
>>>>>         for services such as users, procs, and swap, but not for
>>>>>         ping4 nor ping6, even though I can see the performance data
>>>>>         under "Service State Information".
>>>>>         There are also no rrd files created for these services under
>>>>>         /var/lib/pnp4nagios/perfdata/localhost
>>>>>
>>>>
>>>>         Verify that these check commands and plugins provide valid
>>>>         performance data.
>>>>
>>>>
>>>>
>>>>>         Is it a current limitation/bug, or am I doing something wrong?
>>>>>
>>>>
>>>>         Hard to tell with that little details you provided.
>>>>
>>>>
>>>>>         Please let me know if you require any additional details
>>>>>         from my end. I followed this guide in order to get pnp up
>>>>>         and running:
>>>>>         https://wiki.icinga.org/display/howtos/Setting+up+PNP+
>>>>> with+Icinga2+on+Debian
>>>>>
>>>>>         Thanks in advance.
>>>>>
>>>>>
>>>>>         _______________________________________________
>>>>>         icinga-users mailing list
>>>>>         icinga-users at lists.icinga.org  <mailto:icinga-users at lists.
>>>>> icinga.org>
>>>>>         https://lists.icinga.org/mailman/listinfo/icinga-users
>>>>>
>>>>
>>>>
>>>>         --
>>>>         Michael Friedrich, DI (FH)
>>>>         Application Developer
>>>>
>>>>         NETWAYS GmbH | Deutschherrnstr. 15-19 | D-90429 Nuernberg
>>>>         Tel: +49 911 92885-0 | Fax: +49 911 92885-77
>>>>         <tel:%2B49%20911%2092885-77>
>>>>
>>>>         GF: Julian Hein, Bernd Erk | AG Nuernberg HRB18461
>>>>         http://www.netways.de | Michael.Friedrich at netways.de
>>>>         <mailto:Michael.Friedrich at netways.de>
>>>>
>>>>
>>>>         ** Open Source Backup Conference 2014 - September -
>>>>         osbconf.org <http://osbconf.org> **
>>>>
>>>>         ** Puppet Camp Duesseldorf 2014 - Oktober -
>>>>         netways.de/puppetcamp <http://netways.de/puppetcamp> **
>>>>
>>>>         ** OSMC 2014 - November - netways.de/osmc
>>>>         <http://netways.de/osmc> **
>>>>
>>>>         ** OpenNebula Conf 2014 - Dezember - opennebulaconf.com
>>>>         <http://opennebulaconf.com> **
>>>>
>>>>
>>>>         _______________________________________________
>>>>         icinga-users mailing list
>>>>         icinga-users at lists.icinga.org
>>>>         <mailto:icinga-users at lists.icinga.org>
>>>>
>>>>         https://lists.icinga.org/mailman/listinfo/icinga-users
>>>>
>>>>
>>>>
>>>>
>>>>     _______________________________________________
>>>>     icinga-users mailing list
>>>>     icinga-users at lists.icinga.org  <mailto:icinga-users at lists.
>>>> icinga.org>
>>>>     https://lists.icinga.org/mailman/listinfo/icinga-users
>>>>
>>>
>>>
>>>     --
>>>     Michael Friedrich, DI (FH)
>>>     Application Developer
>>>
>>>     NETWAYS GmbH | Deutschherrnstr. 15-19 | D-90429 Nuernberg
>>>     Tel: +49 911 92885-0 | Fax: +49 911 92885-77
>>>     <tel:%2B49%20911%2092885-77>
>>>
>>>     GF: Julian Hein, Bernd Erk | AG Nuernberg HRB18461
>>>     http://www.netways.de | Michael.Friedrich at netways.de
>>>     <mailto:Michael.Friedrich at netways.de>
>>>
>>>
>>>     ** Open Source Backup Conference 2014 - September - osbconf.org
>>>     <http://osbconf.org> **
>>>
>>>     ** Puppet Camp Duesseldorf 2014 - Oktober - netways.de/puppetcamp
>>>     <http://netways.de/puppetcamp> **
>>>     ** OSMC 2014 - November - netways.de/osmc <http://netways.de/osmc>
>>> **
>>>
>>>     ** OpenNebula Conf 2014 - Dezember - opennebulaconf.com
>>>     <http://opennebulaconf.com> **
>>>
>>>     _______________________________________________
>>>     icinga-users mailing list
>>>     icinga-users at lists.icinga.org <mailto:icinga-users at lists.icinga.org>
>>>
>>>     https://lists.icinga.org/mailman/listinfo/icinga-users
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> icinga-users mailing list
>>> icinga-users at lists.icinga.org
>>> https://lists.icinga.org/mailman/listinfo/icinga-users
>>>
>>
>>
>> --
>> Michael Friedrich, DI (FH)
>> Application Developer
>>
>> NETWAYS GmbH | Deutschherrnstr. 15-19 | D-90429 Nuernberg
>> Tel: +49 911 92885-0 | Fax: +49 911 92885-77
>> GF: Julian Hein, Bernd Erk | AG Nuernberg HRB18461
>> http://www.netways.de | Michael.Friedrich at netways.de
>>
>> ** Open Source Backup Conference 2014 - September - osbconf.org **
>> ** Puppet Camp Duesseldorf 2014 - Oktober - netways.de/puppetcamp **
>> ** OSMC 2014 - November - netways.de/osmc **
>> ** OpenNebula Conf 2014 - Dezember - opennebulaconf.com **
>> _______________________________________________
>> icinga-users mailing list
>> icinga-users at lists.icinga.org
>> https://lists.icinga.org/mailman/listinfo/icinga-users
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.icinga.org/pipermail/icinga-users/attachments/20140810/aa1a35ea/attachment-0001.html>


More information about the icinga-users mailing list