14:07:56 <dfarrell07> #startmeeting Test and Perf
14:07:56 <collabot> Meeting started Thu Jul 28 14:07:56 2016 UTC.  The chair is dfarrell07. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:07:56 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:07:56 <collabot> The meeting name has been set to 'test_and_perf'
14:08:32 <dfarrell07> #info bryan_att raises agenda item about API test coverage in CI, has sent email about it, requests feedback
14:09:14 <dfarrell07> #info Bottleneck is supposed to be leading this meeting, with Yardstick taking notes (neither seem to be here)
14:09:41 <dfarrell07> #info mbeierl has moved wiki pages he was #action'd to move
14:09:53 <kubi001> I 'm from yardstick
14:10:31 <dfarrell07> #info kubi001 from Yardstick is here
14:10:36 <dfarrell07> kubi001: cool, help me take minutes :)
14:10:54 <dfarrell07> #info bryan_att gives overview of OPNFV CI, that there are many APIs exposed
14:11:18 <mbeierl> dfarrell07: chair kubi001?
14:11:30 <dfarrell07> you don't need chair to take minutes FYU
14:11:34 <dfarrell07> you don't need chair to take minutes FYI*
14:11:38 <mbeierl> ah, ok
14:11:48 <kubi001> ok
14:13:44 <dfarrell07> #info bryan_att talks about how ODL, O/S, etc expose APIs. As things change in those APIs, they may break things that depend on them. As we make this deploy more automated, we need to ID when a commit breaks something.
14:14:07 <dfarrell07> #info bryan_att so need to know test coverage over APIs
14:15:15 <kubi001> dfarrell07: thanks for your help. I'm not good at minutes.
14:15:20 <dfarrell07> #info dmcbride talks about how our functional test coverage is doing this, but there's no formal org to put it together except OPNFV
14:15:24 <dfarrell07> kubi001: np :)
14:16:43 <mbeierl> #info dfarrell07 points out the use of Sonar in ODL
14:18:52 <dfarrell07> #info bryan_att example: we can add something to OPNFV that uses O/S API, if that API changes we see breakage in our tests
14:20:59 <dfarrell07> #info mbeierl points out that it may be out of scope for OPNFV to verify all API changes
14:21:31 <mtahhan> #info Maryam Tahhan
14:21:39 <mtahhan> sorry I'm late to the party
14:23:21 <dfarrell07> #info dfarrell07 points out that we wouldn't need to do this API testing except for major releases, as there should be no API changes in daily builds
14:23:33 <dfarrell07> #info so OPNFV doesn't need to do API tests frequently in CI
14:24:43 <dfarrell07> #info dmcbride points out that there are tons of test coverage tools, so may be good to add those to Functest/Yardstick eventually
14:25:58 <dfarrell07> #info mtahhan points out that there have been past discussions about test matrix for coverage, we're not meeting that yet, also concerns about how meaningful test coverage metrics would be in SDN/NFV
14:27:22 <dfarrell07> #info bryan_att says out that as we get more agile, we need more trust in things under us, so more confidence of API coverage
14:27:41 <dfarrell07> #info mbeierl points out that we'd hope we get some of that trust from upstreams
14:28:44 <dfarrell07> #info bryan_att talks about creating wiki to document how testing works for upstreams, what tests they do, where to give coverage info
14:30:18 <Alexander_Victor> Hi, I�m new in NFV and I�m searching a startpoint on it.
14:30:35 <Alexander_Victor> Someone can help me on it ?
14:32:08 <dfarrell07> #info mtahhan redirects discussion back to test vector matrix we've worked on in the past, +1 from dfarrell07
14:34:37 <dfarrell07> #info dmcbride points out that we're in the world of "do you have functional tests, project?", would love to be at "how good are your tests", but not there
14:45:56 <mbeierl> #info Discussion of what the verify, merge, etc are within Gerrit.  Also, questioned how well documented this is within releng.
14:47:15 <mbeierl> #info VSPerf gives example of use of pylint for static code analysis as gate
14:51:05 <mbeierl> #link https://build.opnfv.org/ci/view/storperf/job/storperf-verify-master/ Example of code coverage in Jenkins
14:51:25 <dfarrell07> #info More discussion about documenting tools we use
14:51:33 <dfarrell07> #action mbeierl
14:51:36 <dfarrell07> #char mbeierl
14:51:41 <dfarrell07> #undo
14:51:41 <collabot> Removing item from minutes: <MeetBot.ircmeeting.items.Action object at 0x2e9a850>
14:51:45 <dfarrell07> #chair mbeierl
14:51:45 <collabot> Current chairs: dfarrell07 mbeierl
14:51:47 <dfarrell07> sorry
14:52:12 <mbeierl> #action mbeierl to create a wiki on OPNFV testing and Gerrit hooks options
14:52:23 <dfarrell07> #link http://docs.opendaylight.org/en/latest/submodules/releng/builder/docs/index.html ODL's infra/Jenkins/tools docs
14:59:08 <mbeierl> #action kubi001 to send out information on how to store data in the community Influx DB.
14:59:17 <kubi001> ok
14:59:57 <mbeierl> #info Test projects that have a lot of data can retain their own detail (ie: StorPerf has GBs of details) and report summaries to community DBS
15:00:48 <mbeierl> ending meeting in 10s....
15:01:25 <mbeierl> #endmeeting