#opnfv-testperf: Testing weekly meeting December 15th

Meeting started by morgan_orange at 15:00:50 UTC (full logs).

Meeting summary

  1. call role (morgan_orange, 15:00:56)
    1. Morgan Richomme (morgan_orange, 15:01:01)
    2. kubi (kubi001, 15:01:08)
    3. Mark Beierl (mbeierl, 15:01:09)
    4. Trevor Cooper (trevor_intel, 15:01:24)
    5. agenda https://wiki.opnfv.org/display/meetings/TestPerf (morgan_orange, 15:02:07)

  2. Action point follow up (morgan_orange, 15:03:34)
    1. http://ircbot.wl.linuxfoundation.org/meetings/opnfv-testperf/2016/opnfv-testperf.2016-12-01-15.00.html (morgan_orange, 15:03:44)
    2. no action point (morgan_orange, 15:04:00)

  3. Feedback from the plugfest (morgan_orange, 15:04:14)
    1. more people since #1 (morgan_orange, 15:05:10)
    2. significat improvement: deployments went much better, almost without issues (jose_lausuch, 15:05:19)
    3. installation ready quicker by tuesday most of the installations were completed (morgan_orange, 15:05:34)
    4. Titanium installation from WindRiver (jose_lausuch, 15:05:57)
    5. problems with v3 auth mechanism (jose_lausuch, 15:06:24)
    6. non OPNFV installation challenging but interesting.. (morgan_orange, 15:06:24)
    7. Storperf managed to get it working with HEAT (jose_lausuch, 15:06:40)
    8. Functest managed to fix v3 authentication, but Rally failed. JIRAs created and being fixed by Helen (jose_lausuch, 15:07:19)
    9. https://gerrit.opnfv.org/gerrit/#/c/25725/12 (mbeierl, 15:08:36)
    10. the proposal from Functest to change authentication to v3 (mbeierl, 15:08:52)
    11. no so much tests on Colorado...wotk with Titatnium due to Newton so more a hackfest than a plugfest... (morgan_orange, 15:09:06)
    12. Results collected to the DB. Issues indentified and posted in the internal plugfest wiki (jose_lausuch, 15:10:31)
    13. more remote PODs for the plugfest, easier than shipping (morgan_orange, 15:10:40)
    14. plugfest had more remote hardware this time, more VPN and remote POD access. From Intel it worked well, easier than shipping pods (mbeierl, 15:10:43)
    15. lots of discussions/meetings (CVP, board, Release, ....) (morgan_orange, 15:11:24)
    16. a lot of focus on Infra meetings (pod descriptor, ci evolution...) (jose_lausuch, 15:11:49)
    17. use Dovetail proposal about test descriptor file (jose_lausuch, 15:13:02)
    18. lots of people from Open-O (morgan_orange, 15:14:29)
    19. Interest in Open-O integration in Functest (mbeierl, 15:14:44)

  4. discussion testing landing page (morgan_orange, 15:15:49)
    1. corresponds to one of the Testing priorities in order to give consistency and give a global view (morgan_orange, 15:16:32)
    2. used to be done manually on the wiki in Colorado (hard to maintain and reflect the reality) (morgan_orange, 15:16:53)
    3. proto with functest and yardstick but configurable to welcome other test projects (morgan_orange, 15:17:10)
    4. several filters (installers/version/test projects/ci-loop) (morgan_orange, 15:18:30)
    5. question use of Jenins APi to check deployment result? (morgan_orange, 15:20:44)
    6. question if Daisy is going to be advanced/stable enough to include in dashboard? (mbeierl, 15:26:22)
    7. dashboard code is being stored in releng (mbeierl, 15:30:17)

  5. Performance & Stress tests discussion (morgan_orange, 15:30:29)
    1. question which stress test for Danube (morgan_orange, 15:32:46)
    2. Bottleneck initiated some work on stress tests (morgan_orange, 15:32:57)
    3. yardstick can configure some testing in // now (morgan_orange, 15:33:17)
    4. ACTION: yuyang_ plan a Bottleneck update for the testing community beginning of January (morgan_orange, 15:37:04)
    5. trevor_intel mentioned that the stress test must be easy to reproduce (morgan_orange, 15:37:28)
    6. we need first to organize at a high level on objectives, there are many types of stress and we need to be sure we are using the same terms (bryan_att, 15:40:54)
    7. ACTION: trevor_intel create an etherpad page on stress test: with short term view (Danube) and longer view (stress test strategy) (morgan_orange, 15:44:01)
    8. e.g. resource limits of various types; chaos; multiple operations that might result in some resource conflicts; HA-related stress; .... etc (bryan_att, 15:45:13)

  6. VNF testing (on boarding/ load / stress /) (morgan_orange, 15:46:37)
    1. the key is what we are wanting to achieve and how we will measure it. stress tests are not always binary in terms of success/fail assessment (bryan_att, 15:46:40)
    2. I have several tests already for VNF deployment in Models and we can use them in this stress test if needed (bryan_att, 15:47:27)
    3. again, is this testing just for the purpose of ensuring that MANO functions work, or are we using the MANO stuff as a tool to deploy and test VNFs of significantly different types... the latter is where the Models project can also provide help, as we develop models for various types of VNFs that exercise significantly different types of NFVI resources (bryan_att, 15:49:55)
    4. the Danube goal for Models is to have coverage for all the VNFMs supported in this release, with the same or very similar VNF blueprints, so that may be useful here; e.g. you don't need the full Open-O stack (dozens of docker containers) to drive the system using their NFVO/VNFM (bryan_att, 15:51:50)
    5. we have even fully defined what VNF onboarding is... that's a discussion still underway in the MANO WG (bryan_att, 15:52:51)
    6. what you *can* get out of such testing is to know that the NFVO/VNFM is interoperable with the OPNFV system, as it successfully deploys the VNF and tests of the VNF succeed - even if you don't know what APIs the VNFM etc use (e.g. Heat vs Neutron/Nova), you will know that it is compatible (bryan_att, 15:54:34)
    7. ACTION: morgan_orange cretae a page for vnfOnBoarding for testing (morgan_orange, 15:55:04)

  7. Update on test API testcase collection update (morgan_orange, 15:55:23)
    1. http://testresults.opnfv.org/test/api/v1/projects/functest/cases (morgan_orange, 15:55:45)
    2. any comments welcome (morgan_orange, 15:57:51)

  8. documentation (morgan_orange, 16:00:36)
    1. we need to create an umbrella project (morgan_orange, 16:00:47)
    2. do you agree with the umbrella project (starting end of January) (morgan_orange, 16:01:09)
    3. AGREED: (morgan_orange, 16:01:12)
    4. AGREED: (mbeierl, 16:01:31)
    5. AGREED: (kubi001, 16:01:35)
    6. AGREED: (morgan_orange, 16:01:37)
    7. AGREED: (mbeierl, 16:01:42)

  9. AoB (morgan_orange, 16:01:56)
    1. weekly meetings canceled until end of Decemeber, see you back in January (morgan_orange, 16:03:18)


Meeting ended at 16:04:12 UTC (full logs).

Action items

  1. yuyang_ plan a Bottleneck update for the testing community beginning of January
  2. trevor_intel create an etherpad page on stress test: with short term view (Danube) and longer view (stress test strategy)
  3. morgan_orange cretae a page for vnfOnBoarding for testing


Action items, by person

  1. morgan_orange
    1. morgan_orange cretae a page for vnfOnBoarding for testing
  2. trevor_intel
    1. trevor_intel create an etherpad page on stress test: with short term view (Danube) and longer view (stress test strategy)


People present (lines said)

  1. morgan_orange (45)
  2. mbeierl (12)
  3. jose_lausuch (10)
  4. bryan_att (9)
  5. kubi001 (9)
  6. collabot (6)
  7. trevor_intel (2)


Generated by MeetBot 0.1.4.