#opnfv-meeting: Test and Perf

Meeting started by dfarrell07 at 14:07:56 UTC (full logs).

Meeting summary

    1. bryan_att raises agenda item about API test coverage in CI, has sent email about it, requests feedback (dfarrell07, 14:08:32)
    2. Bottleneck is supposed to be leading this meeting, with Yardstick taking notes (neither seem to be here) (dfarrell07, 14:09:14)
    3. mbeierl has moved wiki pages he was #action'd to move (dfarrell07, 14:09:41)
    4. kubi001 from Yardstick is here (dfarrell07, 14:10:31)
    5. bryan_att gives overview of OPNFV CI, that there are many APIs exposed (dfarrell07, 14:10:54)
    6. bryan_att talks about how ODL, O/S, etc expose APIs. As things change in those APIs, they may break things that depend on them. As we make this deploy more automated, we need to ID when a commit breaks something. (dfarrell07, 14:13:44)
    7. bryan_att so need to know test coverage over APIs (dfarrell07, 14:14:07)
    8. dmcbride talks about how our functional test coverage is doing this, but there's no formal org to put it together except OPNFV (dfarrell07, 14:15:20)
    9. dfarrell07 points out the use of Sonar in ODL (mbeierl, 14:16:43)
    10. bryan_att example: we can add something to OPNFV that uses O/S API, if that API changes we see breakage in our tests (dfarrell07, 14:18:52)
    11. mbeierl points out that it may be out of scope for OPNFV to verify all API changes (dfarrell07, 14:20:59)
    12. Maryam Tahhan (mtahhan, 14:21:31)
    13. dfarrell07 points out that we wouldn't need to do this API testing except for major releases, as there should be no API changes in daily builds (dfarrell07, 14:23:21)
    14. so OPNFV doesn't need to do API tests frequently in CI (dfarrell07, 14:23:33)
    15. dmcbride points out that there are tons of test coverage tools, so may be good to add those to Functest/Yardstick eventually (dfarrell07, 14:24:43)
    16. mtahhan points out that there have been past discussions about test matrix for coverage, we're not meeting that yet, also concerns about how meaningful test coverage metrics would be in SDN/NFV (dfarrell07, 14:25:58)
    17. bryan_att says out that as we get more agile, we need more trust in things under us, so more confidence of API coverage (dfarrell07, 14:27:22)
    18. mbeierl points out that we'd hope we get some of that trust from upstreams (dfarrell07, 14:27:41)
    19. bryan_att talks about creating wiki to document how testing works for upstreams, what tests they do, where to give coverage info (dfarrell07, 14:28:44)
    20. mtahhan redirects discussion back to test vector matrix we've worked on in the past, +1 from dfarrell07 (dfarrell07, 14:32:08)
    21. dmcbride points out that we're in the world of "do you have functional tests, project?", would love to be at "how good are your tests", but not there (dfarrell07, 14:34:37)
    22. Discussion of what the verify, merge, etc are within Gerrit. Also, questioned how well documented this is within releng. (mbeierl, 14:45:56)
    23. VSPerf gives example of use of pylint for static code analysis as gate (mbeierl, 14:47:15)
    24. https://build.opnfv.org/ci/view/storperf/job/storperf-verify-master/ Example of code coverage in Jenkins (mbeierl, 14:51:05)
    25. More discussion about documenting tools we use (dfarrell07, 14:51:25)
    26. ACTION: mbeierl to create a wiki on OPNFV testing and Gerrit hooks options (mbeierl, 14:52:12)
    27. http://docs.opendaylight.org/en/latest/submodules/releng/builder/docs/index.html ODL's infra/Jenkins/tools docs (dfarrell07, 14:52:23)
    28. ACTION: kubi001 to send out information on how to store data in the community Influx DB. (mbeierl, 14:59:08)
    29. Test projects that have a lot of data can retain their own detail (ie: StorPerf has GBs of details) and report summaries to community DBS (mbeierl, 14:59:57)


Meeting ended at 15:01:25 UTC (full logs).

Action items

  1. mbeierl to create a wiki on OPNFV testing and Gerrit hooks options
  2. kubi001 to send out information on how to store data in the community Influx DB.


Action items, by person

  1. kubi001
    1. kubi001 to send out information on how to store data in the community Influx DB.
  2. mbeierl
    1. mbeierl to create a wiki on OPNFV testing and Gerrit hooks options


People present (lines said)

  1. dfarrell07 (31)
  2. mbeierl (11)
  3. collabot (5)
  4. kubi001 (4)
  5. mtahhan (2)
  6. Alexander_Victor (2)


Generated by MeetBot 0.1.4.