[icinga-devel] Icinga Core API

Michael Friedrich michael.friedrich at univie.ac.at
Wed Jun 29 18:51:46 CEST 2011


Hi,

Gunnar Beutner wrote:
> Hello,
>
> as requested by Michael Friedrich I'm pushing the discussion that's so
> far taken place on our internal team list to the icinga-devel list. For
> anyone new to join this discussion here's a short introduction to what
> this is all about:

i do think that such things are way off the hook to be kept internal. 
those are major changes to be discussed, and not just implemented and 
thrown at someone.


>
> During the Icinga meet-up last weekend we've discussed some ideas for an
> Icinga Core API and a way to deal with scalability issues in large-scale
> monitoring setups.

i wasn't taking part during the meeting, so my opinion only reflects 
what i was reading and reflecting in the past 3 days.


those things are 2 seperated portions to take care of.

1) a core api

* adding things like mklivestatus already proposed (GET)
* allowing the SET/POST bidirectional communication
* don't introduce external dependencies
* providing compatibility to existing addons

2) a distributed message queue system

* can be done either as (neb) module (mod_gearman!) or on the new core 
api layer
* stays independent from the existing setup, can be provided "as is" and 
loaded if demanded
* does the internal message queueing itsself and what might be on the 
top layer
* doesn't bypass the core api

>
> The basic idea revolves around using a message queue to decouple
> components like service checks and API methods from the Icinga Core
> while providing a unified interface to these components that can be used
> either locally (i.e. in-process or via IPC) or remotely (via TCP). For
> now we've been thinking about using ZeroMQ for this which is an
> embeddable, (mostly) platform-agnostic message queue library written in C.

i'd love to hear opinions why exactly this should be zeromq and not 
rabbitmq or other candicates like described here e.g.
http://wiki.secondlife.com/wiki/Message_Queue_Evaluation_Notes


> You can find an updated in-depth description of the proposed changes
> involved in the MQ implementation at
> https://wiki.icinga.org/display/Dev/Ideas+for+new+Core+API . I'd love to
> hear your feedback about the design specifications which for now are
> primarily being worked on by Ricardo and me.

summarizing the points i've already posted in recent internal mails,

* the newly introduced core api should be used as poc for classicui and 
integrated live search, providing simple GET hosts in json and nothing 
more basically


* the libicinga is on top of the mq architecture, which then remains 
rather independant of the underlaying core architecture. so whatever you 
are wrapping around, you could still do it a layer down below, if you 
know what you are doing .... "libicinga is built on zeromq and people 
tend to use other technologies - like gearman or livestatus wrappers for 
nagvis, nagiosbp and such. the idea is *not* to reinvent the wheel, but 
stay compatible to existing solutions. addons devs won't jump onto that 
just because it's cool and costs dev time."


* the proof of concept for distributed monitoring should take place with 
the current neb architecture .... "proof of concept as core module - 
take a look at mod_gearman, and replace that parts with zeromq, realize 
the proof of concept as neb module with NEB_CALLBACK_OVERRIDE or 
NEB_CALLBACK_CANCEL and threadinng locks."


* the core api should be extendable. not only for icingamq but also for ...

** idoutils replacement, with a cached worker

** add configs via api, store internally, provide a config editor (or at 
least a community addon) like demanded over 
http://feedback.icinga.org/forums/50329-general/suggestions/754152-api-to-add-new-hosts-groups-services-commands-auto?ref=title

[--SNIP--]

1) design a core api capable of that (icingamq doesn't include that by
default as far as i have read)
2) allow simple configs to be added into the core, and still written
somewhere
3) adopt a config data layout
4) build a custom config web onto that
5) integrate that web in classicui and icingaweb

[--SNIP--]

* the core api as well as the icingamq design *must* entirely stay out 
of the current 1.x tree and probably provided as package to be tested, 
but not provided officially (and also not supported).
and tbh, this would be something for icinga 2.x where new features can 
be introduced and some show stoppers can be deprecated whilst 
introducing new things.


* the message format for icingamq must be discussed either on 
icinga-devel or icinga-users mailinglists (or a public poll if there was 
a decision), allowing the community to add their ideas to a general RFC 
- which needs to be written in the first place.


* evaluate the core logic on the eventloop block for api fetches, or 
threaded synchronisation. this includes feeding checkresults e.g.


* keep the code clean, follow the style guide and developer guidelines. 
this includes doxygen and also further documentation / rfcs.



kind regards,
michael

-- 
DI (FH) Michael Friedrich

Vienna University Computer Center
Universitaetsstrasse 7 A-1010 Vienna, Austria

email: 	michael.friedrich at univie.ac.at
phone: 	+43 1 4277 14359
mobile: +43 664 60277 14359
fax: 	+43 1 4277 14338
web:	http://www.univie.ac.at/zid
	http://www.aco.net

Icinga Core&  IDOUtils Developer
http://www.icinga.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.icinga.org/pipermail/icinga-devel/attachments/20110629/df7916d1/attachment.html>


More information about the icinga-devel mailing list