#opnfv-meeting: Testing community weekly meeting July 7th

Meeting summary

1. call role (morgan_orange, 13:59:37)
   1. Morgan Richomme (Functest) (morgan_orange, 13:59:45)
   2. Chris Price (lurking IRC only) (ChrisPriceAB, 14:00:06)
   3. kubi (kubi001, 14:00:21)
   4. Jose Lausuch (jose_lausuch, 14:03:04)
   
   
2. Colorado status (morgan_orange, 14:04:12)
   1. status for yardstick (morgan_orange, 14:05:23)
   2. https://wiki.opnfv.org/display/yardstick/Yardstick+Colorado+Release+Planning (morgan_orange, 14:05:25)
   3. https://etherpad.opnfv.org/p/yardstick_release_c_feature_freeze (kubi001, 14:06:39)
   4. Jing (Jing_, 14:07:03)
   5. status Functest (morgan_orange, 14:07:06)
   6. integration phase in progress: copper, moon, security_scan, sfc-odl, sfc-onos (morgan_orange, 14:07:35)
   7. status Vsperf (morgan_orange, 14:08:21)
   8. everything should be in the repo :) interaction with yardstick project for integration, no major concerns (morgan_orange, 14:09:23)
   
   
3. release criteria (morgan_orange, 14:10:38)
   1. easy for functional testing.. (PASS/FAIL) (morgan_orange, 14:10:49)
   2. not easy for performance/qualification, KPI not easy to define (morgan_orange, 14:11:12)
   3. for performance tests (yardstick/vsperf) if we can get figure, we will consider the test as PASS (morgan_orange, 14:12:09)
   4. Functest criteria in the test case => automatic reporting (morgan_orange, 14:25:43)
   5. http://testresults.opnfv.org/reporting/functest/release/master/index-status-fuel.html (morgan_orange, 14:25:48)
   6. code of automatic reporting is in the releng repo (morgan_orange, 14:26:26)
   7. https://git.opnfv.org/cgit/releng/tree/utils/test/reporting (morgan_orange, 14:26:29)
   8. ELK and influxDB and grafan will coexist (morgan_orange, 14:26:47)
   9. you can use both solutions, it depends on your usage and what you want to visualize (morgan_orange, 14:27:02)
   10. question on the 4 consecutive runs: concerns from the infra group (morgan_orange, 14:36:11)
   11. concerns because we experienced in Brahmaputra that simple Functional tests may fail/pass without one line change => concern on the robustness (morgan_orange, 14:36:42)
   12. side effects due to complex configuration may lead to strange behavior and inconsistancy in test results (morgan_orange, 14:37:16)
   13. ACTION: dmcbride: organize a sync meeting Infra/testing (Daniel chairing the topic) to discuss this topic (morgan_orange, 14:39:56)
   
   
4. documentation (morgan_orange, 14:43:18)
   1. need for OPNFV testing = overall picture of all the test projects (morgan_orange, 14:52:43)
   2. ~ conversation with the community (morgan_orange, 14:52:53)
   3. Dave Urschatz (durschatz, 14:53:18)
   4. ACTION: mbeierl dmcbride invite Alan for a status on testing whitepaper (morgan_orange, 14:57:19)
   5. Dave ask management if possible to coordinate the redaction of such document (morgan_orange, 14:58:32)
   6. ACTION: all PTLs: ask preference for meeting time (allow Asian contributors to join) (morgan_orange, 15:00:48)
   7. Add Yardstick test evolution to agenda for next week (mbeierl, 15:01:44)

Action items

1. dmcbride: organize a sync meeting Infra/testing (Daniel chairing the topic) to discuss this topic
2. mbeierl dmcbride invite Alan for a status on testing whitepaper
3. all PTLs: ask preference for meeting time (allow Asian contributors to join)

Action items, by person

1. dmcbride
   1. dmcbride: organize a sync meeting Infra/testing (Daniel chairing the topic) to discuss this topic
   2. mbeierl dmcbride invite Alan for a status on testing whitepaper
2. mbeierl
   1. mbeierl dmcbride invite Alan for a status on testing whitepaper

People present (lines said)

1. morgan_orange (35)
2. collabot` (5)
3. kubi001 (4)
4. ChrisPriceAB (2)
5. mbeierl (2)
6. Jing_ (1)
7. durschatz (1)
8. jose_lausuch (1)
9. dmcbride (1)
10. yujunz (1)