17:01:36 <tbachman> #startmeeting tws
17:01:36 <odl_meetbot> Meeting started Mon Jul 13 17:01:36 2015 UTC.  The chair is tbachman. Information about MeetBot at http://ci.openstack.org/meetbot.html.
17:01:36 <odl_meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:01:36 <odl_meetbot> The meeting name has been set to 'tws'
17:01:39 <tbachman> #chair alagalah
17:01:39 <odl_meetbot> Current chairs: alagalah tbachman
17:01:41 <tbachman> #chair colindixon
17:01:41 <odl_meetbot> Current chairs: alagalah colindixon tbachman
17:01:50 <tbachman> #topic SDN SDK Presentation
17:02:09 * colindixon wanders in
17:02:24 <alagalah> colindixon: I don't see anyone from SDN SDK team yet
17:03:09 <alagalah> colindixon: Still no Partha
17:03:14 <sumit_kapoor> hello everyone
17:03:40 <icbts> Howdy
17:06:01 <tbachman> #info The SDN SDK group had a creation review, and the TSC had a lot of questions around the project’s scope. This TWS was meant to provide a forum to allow the SDN SDK group to better address/present the scope of their project
17:06:47 <tbachman> #info The focus of the Centinal project is only for the logging functionality of ODL
17:07:17 <tbachman> #info The ODL logging mechanism is very expensive — multiple logs generated from various ODL features, modules, and tools
17:07:36 <tbachman> #info Engineers are expected to have ODL know-how to figure out seach queries in logs to extract intelligent data
17:07:57 <abhijitkumbhare> Sorry - I believe I may have held up Partha and folks in the OpenFlow plugin meeting
17:07:59 <tbachman> #info The isn’t a real-time tracking of logs (per user defined rules) for specific error conditions or diagnosis
17:08:03 <abhijitkumbhare> Over now
17:08:15 <Prem_> Interesting that, my topic for ODL summit is the same :)
17:08:19 <abhijitkumbhare> actually over 3-4 minutes ago
17:08:30 <tbachman> #info expcetation of a classid monitoring user-interface to track set rules/threshold alerts log/data
17:08:51 <tbachman> Prem_: they’re saving you some work :)
17:08:55 <dbainbri> Is logging at the classID level really the right level for clients of SDN
17:09:03 <dbainbri> that is more development focused.
17:09:05 <Prem_> Already effort has gone in to integrate with fluentd
17:09:16 <dbainbri> what about logging at a business level or above
17:09:33 <tbachman> dbainbri: Prem_ are you guys on the call?
17:09:55 <dbainbri> tbachman: not yet, finishing up another call, joining soon.
17:09:56 <Prem_> Yes
17:10:08 <tbachman> #info The project uses the MD-SAL, Log source, DB/Presistence, cassandra/Hbase from ODL
17:10:37 <tbachman> #info it Adds a web interface with dashboard, alerts, streams, log search and analytics, diagnostics, and health
17:10:51 <tbachman> #info There will also be NB APIs
17:11:05 <dbainbri> what about something like an ELK stack for logging as opposed to build our own.
17:11:09 <tbachman> #info Prem_ has submitted a similar topic for the ODL summit
17:11:16 <tbachman> #undo
17:11:16 <odl_meetbot> Removing item from minutes: <MeetBot.ircmeeting.items.Info object at 0x1c3d110>
17:11:31 <tbachman> #info Prem_ has submitted a similar topic for the ODL summit, and is interested in possibly collaborating with this effort
17:12:02 <tbachman> #info Prem_ proposal includes a log analyzer in order to correlate logs between various modules as a first step
17:12:05 <icbts> You might want to look at https://github.com/savoirtech/hecate for a Karaf-Cassandra library to handle volume of requests you’ll be making
17:12:32 <tbachman> #info dbainbri asks why a standard off-the-shelf library/tools (e.g. ELK) insn’t used instead
17:12:51 <tbachman> #info Prem_ says that for their proposal is to use the log stack for available analyzers
17:13:20 <tbachman> #info sumit_kapoor says they’re doing more analysis and features that aren’t provided by current analyzers — things that are ODL specific
17:13:55 <tbachman> #info dbainbri asks if they’re driving all logs through the MD-SAL
17:14:17 <tbachman> #info sumit_kapoor says only the rules are going into the ODL data store
17:15:34 <tbachman> #info dbainbri says sounds more like an analyzer
17:17:36 <tbachman> #info dbainbri asks if he has to update his log4j code to support this
17:17:46 <tbachman> #info sumit_kapoor says you don’t have to update your code
17:18:45 <tbachman> #info sumit_kapoor says there’s a syslog appender that sends the messages in real time to their syslog
17:19:27 <tbachman> #info sumit_kapoor says they offer more analytics/diagnostics before persisting
17:21:05 <tbachman> #Info dbainbri says he sees two different things — log collection and real-time analytics and batch processing (e.g. SPARK), and wouldn’t want the MD-SAL in the way of eitehr of those
17:23:50 <tbachman> #info sumit_kapoor says this is a framework for the performance data as well.
17:25:21 <tbachman> #info dbainbri is concerned about multiple projects going after the same capability
17:29:17 <tbachman> #info moizer asks if they’ve considered using Flume + Kibana for the same purpose?
17:30:25 <tbachman> sumit_kapoor: Prem_: dbainbri: ^^^ see moizer’s q’
17:31:19 <dbainbri> flume +  kibana?
17:31:48 <dbainbri> right now i care less about technology and more about a common philosophy about stream processing
17:32:02 <tbachman> #info Prem_ says their presentation would be around that
17:32:30 <tbachman> #info dbainbri says right now he’s less concerned about technology and more concerned about a common philosophy about stream processing
17:32:51 <dbainbri> #info dbainbri also includes a common direction in batch processing as well.
17:33:19 <dbainbri> (and now will shut up as he thinks he made the point way too many times and too vigorously)
17:33:29 <tbachman> lol
17:33:36 <tbachman> dbainbri: debate is good :)
17:35:26 <tbachman> #info The project provides a streams service, which has a mechanism to route messages into categories in real time while they are processed like stream for audit logs; rule configuration include message, level, source, etc.; streams are generated by the graylog server as per user-defined rules; event handler module handles streams from graylog server and persist them into centinel model
17:36:00 <tbachman> #info Log alerts service allows alerts generated based on specific event matching in real-time; alert condition types can be message count, field value, and field string value
17:36:40 <tbachman> #info Search and Analyze service supports a search query language; a time range for search can be specified; visualization includes histogram
17:39:39 <Prem_> dbainbri:  Would a unified log analyzer that analyzes SDN Apps + ODL + SDN switch be beneficial
17:40:09 <tbachman> #info Prem_ asks dbainbri if a unified log analyzer that analyzes SDN Apps + ODL + SDN switch would be beneficial?
17:41:46 <dbainbri> Prem_: how is log analysis different from analyzing any other stream data? My argument would be it isn't, so that ODL needs a way to analyze stream data and combine data from multiple sources in that analysis.
17:42:21 <dbainbri> Prem_: so log data (as well as other data) is normalized in some way and then people can write different analytics against the data.
17:42:31 <tbachman> #info dbainbri asks how log analysis is different from analyzing any other data stream — argues that it isn’t, so that ODL needs a way to analyze stream data and combine data from multiple sources in that analysis
17:43:01 <tbachman> #info dbainbri says this normalizes log data (as well as other data) in some way, and then people can write different analytics against the data
17:43:10 <dbainbri> #info dbainbri thinks log data is just yet another event
17:43:20 <tbachman> dbainbri: thx :)
17:43:33 * tbachman appreciates help with IRC notes
17:43:51 <dbainbri> #info dbainbri also thinks ODL logs are often clogged with "system or debug" logs as opposed to messages that mean something from the "control" aspect.
17:44:13 <Prem_> dbainbri:  I agree.  It does become another stream of data provided it is processed and normalized
17:44:23 <dbainbri> #info dbainbri continues to think that this means not all logs have to end in a generated event that needs to be analyzed .
17:44:30 <tbachman> #info Prem_ agrees it does become another stream of data provided it is processed and normalized
17:44:47 <tbachman> #info alagalah asks if there’s a new project proposal for Centinel?
17:45:08 <tbachman> #info sumit_kapoor says this is a new project proposal — it’s one of the features of the original SDN SDK proposal
17:46:07 <tbachman> #info alagalah asks if they’re using the persistence project — not introducing a new dependency on the data store
17:46:14 <tbachman> #info sumit_kapoor says they’re using the persistence project
17:46:47 <tbachman> #info dbainbri asks if alarms are going through the persistence layer
17:47:01 <tbachman> #info sumit_kapoor says yes
17:47:47 <tbachman> #info dbainbri asks if the ODL persistence project is capable of handling an alarm storm?
17:47:57 * dbainbri shivers with fright
17:48:08 <tbachman> #info sumit_kapoor says if these things happen, the alarm storm would be handled before persisting
17:50:54 <dbainbri> colindixon: "ask in a different way" translates to "in a much less obnoxious way" :)
17:51:08 <dbainbri> (with clarification context as well)
17:52:12 <tbachman> #info colindixon asks if the project decomposes into 3 pieces; log gathering; log analysis framework; analysis programs
17:54:12 <tbachman> #info colindixon says it’s more work to create rules that make meaningful alarms than it is to create a rules engine — asks if they will be contributing rules themselves
17:54:26 <tbachman> #info sumit_kapoor says they will specify some preconfigured rules — e.g. for the openflowplugin
17:54:39 <Prem_> Can the rules be derived from the state diagram of any apps / module
17:54:42 <tbachman> #info sumit_kapoor says ODL users can creat emore rules
17:54:56 <tbachman> #info Prem_ asks if the rules can be derived from the state diagrams of any apps/module?
17:55:12 <tbachman> #info sumit_kapoor says they will also provide some preconfigured rules for SFC
18:04:26 <dbainbri> #info dbainbri: need to separate technology from common best practice philosophy
18:05:13 <tbachman> #endmeeting