=============================
#opnfv-meeting: Test and Perf
=============================


Meeting started by dfarrell07 at 14:07:56 UTC.  The full logs are
available at
http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-07-28-14.07.log.html
.



Meeting summary
---------------

* bryan_att raises agenda item about API test coverage in CI, has sent
  email about it, requests feedback  (dfarrell07, 14:08:32)
* Bottleneck is supposed to be leading this meeting, with Yardstick
  taking notes (neither seem to be here)  (dfarrell07, 14:09:14)
* mbeierl has moved wiki pages he was #action'd to move  (dfarrell07,
  14:09:41)
* kubi001 from Yardstick is here  (dfarrell07, 14:10:31)
* bryan_att gives overview of OPNFV CI, that there are many APIs exposed
  (dfarrell07, 14:10:54)
* bryan_att talks about how ODL, O/S, etc expose APIs. As things change
  in those APIs, they may break things that depend on them. As we make
  this deploy more automated, we need to ID when a commit breaks
  something.  (dfarrell07, 14:13:44)
* bryan_att so need to know test coverage over APIs  (dfarrell07,
  14:14:07)
* dmcbride talks about how our functional test coverage is doing this,
  but there's no formal org to put it together except OPNFV
  (dfarrell07, 14:15:20)
* dfarrell07 points out the use of Sonar in ODL  (mbeierl, 14:16:43)
* bryan_att example: we can add something to OPNFV that uses O/S API, if
  that API changes we see breakage in our tests  (dfarrell07, 14:18:52)
* mbeierl points out that it may be out of scope for OPNFV to verify all
  API changes  (dfarrell07, 14:20:59)
* Maryam Tahhan  (mtahhan, 14:21:31)
* dfarrell07 points out that we wouldn't need to do this API testing
  except for major releases, as there should be no API changes in daily
  builds  (dfarrell07, 14:23:21)
* so OPNFV doesn't need to do API tests frequently in CI  (dfarrell07,
  14:23:33)
* dmcbride points out that there are tons of test coverage tools, so may
  be good to add those to Functest/Yardstick eventually  (dfarrell07,
  14:24:43)
* mtahhan points out that there have been past discussions about test
  matrix for coverage, we're not meeting that yet, also concerns about
  how meaningful test coverage metrics would be in SDN/NFV  (dfarrell07,
  14:25:58)
* bryan_att says out that as we get more agile, we need more trust in
  things under us, so more confidence of API coverage  (dfarrell07,
  14:27:22)
* mbeierl points out that we'd hope we get some of that trust from
  upstreams  (dfarrell07, 14:27:41)
* bryan_att talks about creating wiki to document how testing works for
  upstreams, what tests they do, where to give coverage info
  (dfarrell07, 14:28:44)
* mtahhan redirects discussion back to test vector matrix we've worked
  on in the past, +1 from dfarrell07  (dfarrell07, 14:32:08)
* dmcbride points out that we're in the world of "do you have functional
  tests, project?", would love to be at "how good are your tests", but
  not there  (dfarrell07, 14:34:37)
* Discussion of what the verify, merge, etc are within Gerrit.  Also,
  questioned how well documented this is within releng.  (mbeierl,
  14:45:56)
* VSPerf gives example of use of pylint for static code analysis as gate
  (mbeierl, 14:47:15)
* LINK:
  https://build.opnfv.org/ci/view/storperf/job/storperf-verify-master/
  Example of code coverage in Jenkins  (mbeierl, 14:51:05)
* More discussion about documenting tools we use  (dfarrell07, 14:51:25)
* ACTION: mbeierl to create a wiki on OPNFV testing and Gerrit hooks
  options  (mbeierl, 14:52:12)
* LINK:
  http://docs.opendaylight.org/en/latest/submodules/releng/builder/docs/index.html
  ODL's infra/Jenkins/tools docs  (dfarrell07, 14:52:23)
* ACTION: kubi001 to send out information on how to store data in the
  community Influx DB.  (mbeierl, 14:59:08)
* Test projects that have a lot of data can retain their own detail (ie:
  StorPerf has GBs of details) and report summaries to community DBS
  (mbeierl, 14:59:57)



Meeting ended at 15:01:25 UTC.



Action items, by person
-----------------------

* kubi001
  * kubi001 to send out information on how to store data in the
    community Influx DB.
* mbeierl
  * mbeierl to create a wiki on OPNFV testing and Gerrit hooks options



People present (lines said)
---------------------------

* dfarrell07 (31)
* mbeierl (11)
* collabot (5)
* kubi001 (4)
* mtahhan (2)
* Alexander_Victor (2)



Generated by `MeetBot`_ 0.1.4