[icinga-users] icinga-users Digest, Vol 37, Issue 10

Vikas Tiwari vikaskt14 at gmail.com
Sat Jan 14 12:23:53 CET 2017


Hi,

IOT device my be is a wrong word. Sorry.

Well, the machines are Hypriot Linux based Raspberry Pies. RPI2 & RPI3.  We
want to measure all the basic things like disk, CPU, Mem, Network, NTP,
docker containers, etc plus other things.

There are a total of 10 checks currently after all the reduction.


On Sat, Jan 14, 2017 at 4:30 PM, <icinga-users-request at lists.icinga.org>
wrote:

> Send icinga-users mailing list submissions to
>         icinga-users at lists.icinga.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://lists.icinga.org/mailman/listinfo/icinga-users
> or, via email, send a message with subject or body 'help' to
>         icinga-users-request at lists.icinga.org
>
> You can reach the person managing the list at
>         icinga-users-owner at lists.icinga.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of icinga-users digest..."
>
>
> Today's Topics:
>
>    1. Re: icinga-users Digest, Vol 37, Issue 9 (Vikas Tiwari)
>    2. Re: icinga-users Digest, Vol 37, Issue 9 (Michael Friedrich)
>    3. Re: icinga-users Digest, Vol 37, Issue 9 (Antony Stone)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 13 Jan 2017 19:56:46 +0530
> From: Vikas Tiwari <vikaskt14 at gmail.com>
> To: icinga-users at lists.icinga.org
> Subject: Re: [icinga-users] icinga-users Digest, Vol 37, Issue 9
> Message-ID:
>         <CAEVQhqpTJ2AgMT7YHb=Po2O+C8+Y1HGLy0kq5TAnSrWhVOph2g at mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Antony,
> Thank you for taking time to reply.
>
> We are monitoring IOT devices. These devices are in remote locations and
> behind one-way NAT. We do not have the luxury of unlimited network and
> network at times can go bad.
>
> *Why passive?*
> *Because I want the replay logs. Many times the connection b/w master and
> client breaks and in such scenarios we want the client to keep running the
> checks and send the check output when the connection re-establishes.*
>
>
> *Is the Master able to connect to the Clients through the NAT connection?
> The fact that you said above that "the clients (behind NAT) connect to the
> the master directly" suggests that it cannot (ie: it sounds like one-way
> NAT), so*
>
> *have you made sure that the Master Endpoint definitions
> in /etc/icinga2/zones.conf on the Clients contains the hostname (or IP
> address) of the Master, but that the Client Endpoint definitions on the
> Master does notcontain the hostname or IP address of the Clients?*
> *No, I dont want the master to connect to client. Yes its a one-way NAT.
> Yes i have made sure that the master endpoint definitions in zones.conf of
> the client contain resolvable hostname and client endpoint definitions on
> the master do not contain the hostname.*
>
> *What bandwidth do you have available per client connection?*
> *I do not want 1 MB per hour. This will cost us 720 MB just for monitoring
> the IOT device. I want the data to be as much less as possible. *
> * Please note that i have already optimised all the checks to output as
> much data as possible.*
>
> *That equals around 256 bits per second (if KB means kilobits) or 2kbits
> per second (if KB means kilobytes).*
> *That is Kilobytes.*
>
> *Is that *really* a significant overhead on your network?*
> *I am very happy to deploy Icinga2 client on an AWS machine and there we
> don't care about the data usage at all. But for an IOT device, is it the
> right tool ? Given i have already given you all the information*
>
> *Changing the heartbeat interval from 10s to 100s and log::SetLogPosition
> interval from 5s to 60s will reduce a significant overhead. But that is
> hard coded. Can we have it as a configurable entity ?*
>
> Good day !
> Vikas
>
> On Fri, Jan 13, 2017 at 4:30 PM, <icinga-users-request at lists.icinga.org>
> wrote:
>
> > Send icinga-users mailing list submissions to
> >         icinga-users at lists.icinga.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >         https://lists.icinga.org/mailman/listinfo/icinga-users
> > or, via email, send a message with subject or body 'help' to
> >         icinga-users-request at lists.icinga.org
> >
> > You can reach the person managing the list at
> >         icinga-users-owner at lists.icinga.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of icinga-users digest..."
> >
> >
> > Today's Topics:
> >
> >    1. Icinga2 reduce network data usage (Vikas Tiwari)
> >    2. Re: Icinga2 reduce network data usage (Antony Stone)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Thu, 12 Jan 2017 18:43:48 +0530
> > From: Vikas Tiwari <vikaskt14 at gmail.com>
> > To: icinga-users at lists.icinga.org
> > Subject: [icinga-users] Icinga2 reduce network data usage
> > Message-ID:
> >         <CAEVQhqrWHmYCxsXGpTVP+fRiOucKsrHFJD3P2pOd4RSsE40weg@
> > mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Hi,
> >
> > I am testing Icinga2 as a monitoring tool for our infrastructure which
> will
> > monitor close to 3000 nodes.
> >
> > I have a top down config sync. There is a single master and and there are
> > no fail overs(for now). The clients(behind NAT) connect to the master
> > directly.
> >
> > I have configured around 20 passive checks on each client. What i have
> > observed is that the heartbeats from both the ends i.e the master and the
> > clients are very aggressive which ping every 10 seconds.
> >
> > Further the JSONRPC pings are as frequent as 5 seconds, there is also a
> lot
> > of other data exchanged between the client and the master. This all leads
> > to high consumption of network data, Given we have limited  data on the
> > client machines we cannot afford to spend this much data just for
> > monitoring.
> >
> > Currently, per hour 800-900 KB is consumed.Out of this only around 300KB
> > seems to be the check output. Check intervals are 6 minutes.
> >
> > The hearbeat and other intervals are hard coded in the source code itself
> > which will be my last option.
> >
> > Is there any other way i can reduce the network data consumption. ?
> >
> > Vikas Tiwari
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: <http://lists.icinga.org/pipermail/icinga-users/
> > attachments/20170112/14ecfd34/attachment-0001.html>
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Thu, 12 Jan 2017 14:58:41 +0100
> > From: Antony Stone <Antony.Stone at icinga.open.source.it>
> > To: "Icinga User's Corner" <icinga-users at lists.icinga.org>
> > Subject: Re: [icinga-users] Icinga2 reduce network data usage
> > Message-ID: <201701121458.42028.Antony.Stone at icinga.open.source.it>
> > Content-Type: Text/Plain;  charset="iso-8859-15"
> >
> > On Thursday 12 January 2017 at 14:13:48, Vikas Tiwari wrote:
> >
> > > Hi,
> > >
> > > I am testing Icinga2 as a monitoring tool for our infrastructure which
> > will
> > > monitor close to 3000 nodes.
> > >
> > > I have a top down config sync. There is a single master and and there
> are
> > > no fail overs(for now). The clients(behind NAT) connect to the master
> > > directly.
> > >
> > > I have configured around 20 passive checks on each client.
> >
> > Why passive?
> >
> > > What i have observed is that the heartbeats from both the ends i.e the
> > > master and the clients are very aggressive which ping every 10 seconds.
> >
> > Is the Master able to connect to the Clients through the NAT connection?
> > The
> > fact that you said above that "the clients (behind NAT) connect to the
> the
> > master directly" suggests that it cannot (ie: it sounds like one-way
> NAT),
> > so
> > have you made sure that the Master Endpoint definitions in
> > /etc/icinga2/zones.conf on the Clients contains the hostname (or IP
> > address)
> > of the Master, but that the Client Endpoint definitions on the Master
> does
> > not
> > contain the hostname or IP address of the Clients?
> >
> > > Further the JSONRPC pings are as frequent as 5 seconds, there is also a
> > lot
> > > of other data exchanged between the client and the master. This all
> leads
> > > to high consumption of network data, Given we have limited  data on the
> > > client machines we cannot afford to spend this much data just for
> > > monitoring.
> >
> > What bandwidth do you have available per client connection?
> >
> > > Currently, per hour 800-900 KB is consumed.
> >
> > That equals around 256 bits per second (if KB means kilobits) or 2kbits
> per
> > second (if KB means kilobytes).
> >
> > Is that *really* a significant overhead on your network?
> >
> >
> > Antony.
> >
> > --
> > I want to build a machine that will be proud of me.
> >
> >  - Danny Hillis, creator of The Connection Machine
> >
> >                                                    Please reply to the
> > list;
> >                                                          please *don't*
> CC
> > me.
> >
> >
> > ------------------------------
> >
> > Subject: Digest Footer
> >
> > _______________________________________________
> > icinga-users mailing list
> > icinga-users at lists.icinga.org
> > https://lists.icinga.org/mailman/listinfo/icinga-users
> >
> >
> > ------------------------------
> >
> > End of icinga-users Digest, Vol 37, Issue 9
> > *******************************************
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.icinga.org/pipermail/icinga-users/
> attachments/20170113/2b22f60a/attachment-0001.html>
>
> ------------------------------
>
> Message: 2
> Date: Fri, 13 Jan 2017 15:36:51 +0100
> From: Michael Friedrich <michael.friedrich at icinga.com>
> To: Icinga User's Corner <icinga-users at lists.icinga.org>
> Subject: Re: [icinga-users] icinga-users Digest, Vol 37, Issue 9
> Message-ID: <E3E120BE-6BCF-4AE0-A2D1-42E0EEAA7846 at icinga.com>
> Content-Type: text/plain; charset=us-ascii
>
> Hi,
>
> > On 13 Jan 2017, at 15:26, Vikas Tiwari <vikaskt14 at gmail.com> wrote:
> >
> > I am very happy to deploy Icinga2 client on an AWS machine and there we
> don't care about the data usage at all. But for an IOT device, is it the
> right tool ?
>
> What kind of IoT device?
>
> Kind regards,
> Michael
>
>
>
>
> ------------------------------
>
> Message: 3
> Date: Fri, 13 Jan 2017 17:25:21 +0100
> From: Antony Stone <Antony.Stone at icinga.open.source.it>
> To: "Icinga User's Corner" <icinga-users at lists.icinga.org>
> Subject: Re: [icinga-users] icinga-users Digest, Vol 37, Issue 9
> Message-ID: <201701131725.21584.Antony.Stone at icinga.open.source.it>
> Content-Type: Text/Plain;  charset="iso-8859-15"
>
> On Friday 13 January 2017 at 15:26:46, Vikas Tiwari wrote:
>
> > Hi Antony,
> > Thank you for taking time to reply.
> >
> > We are monitoring IOT devices. These devices are in remote locations and
> > behind one-way NAT. We do not have the luxury of unlimited network and
> > network at times can go bad.
>
> You call them "IoT devices" - how much are they doing, and how complex is
> the
> monitoring you need to perform on them?
>
> I assume you're not getting a whole set of data such as disk usage, number
> of
> processes, memory usage, network bandwidth... which would be common with
> full
> network client/server type devices.
>
> So, what are you trying to measure about these things?
>
> > > Why passive?
> >
> > Because I want the replay logs. Many times the connection b/w master and
> > client breaks and in such scenarios we want the client to keep running
> the
> > checks and send the check output when the connection re-establishes.
>
> Okay, that sounds reasonable.
>
> > > What bandwidth do you have available per client connection?
> >
> > I do not want 1 MB per hour. This will cost us 720 MB just for monitoring
> > the IOT device. I want the data to be as much less as possible.
> > Please note that i have already optimised all the checks to output as
> > much data as possible.
>
> Er, if you really mean what you just said, that could be your problem?
>
> Surely to reduce bandwidth you want to minimise the data output from the
> checks?
>
> How many checks are you performing on each device?
>
> > > That equals around 256 bits per second (if KB means kilobits) or 2kbits
> > > per second (if KB means kilobytes).
> >
> > That is Kilobytes.
>
> So, 2kbits / second per device is excessive... hm, that's quite a tough
> challenge...
>
> > for an IOT device, is it the right tool ?
>
> Difficult to say - what do *you* mean by "IoT", and what are you trying to
> measure about these things?
>
>
> Antony.
>
> --
> I lay awake all night wondering where the sun went, and then it dawned on
> me.
>
>                                                    Please reply to the
> list;
>                                                          please *don't* CC
> me.
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> icinga-users mailing list
> icinga-users at lists.icinga.org
> https://lists.icinga.org/mailman/listinfo/icinga-users
>
>
> ------------------------------
>
> End of icinga-users Digest, Vol 37, Issue 10
> ********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.icinga.org/pipermail/icinga-users/attachments/20170114/31660394/attachment-0001.html>


More information about the icinga-users mailing list