#opnfv-testperf: OPNFV Test Working Group

Meeting started by dmcbride at 15:03:48 UTC (full logs).

Meeting summary

    1. Al Morton (acm_, 15:04:05)
    2. David McBride (dmcbride, 15:04:12)
    3. Trevor Cooper (trevor_intel, 15:04:16)
    4. (acm_, 15:08:48)
    5. Al Morton (acm_, 15:09:04)
    6. Rex Lee (mj_rex, 15:09:51)
    7. add Akraino testing to the agenda (acm_, 15:10:21)

  2. 2019 working group planning (dmcbride, 15:11:57)
    1. good turn-out today, 9 participants (acm_, 15:17:41)
    2. dmcbride keep meeting - this group is of vital importance to OPNFV (acm_, 15:18:49)
    3. Board input - two asks: OVP and a common NFVI infrastructure (acm_, 15:19:55)
    4. Bin elab on NFVI point: many variations of infrastructure with many scenarios - does not assure interop for service provider customers (acm_, 15:21:12)
    5. dmcbride a clear Board priorty is Testing! (acm_, 15:21:57)
    6. dmcbride trying to overhaul rel process - meetings at plugfest were productive, new ideas for process (acm_, 15:22:57)
    7. dmcbride one idea - specific gates , non-feature scearios define their own (single) gate, for example (acm_, 15:23:47)
    8. dmcbride this supports more continuous release (above) (acm_, 15:24:33)
    9. dmcbride for feature releases - have as many as 3 or 4 gates - Trevor Bramwell proposed that TSC takes Recommendation from Testing WG for gate design (acm_, 15:25:40)
    10. Rec includes: number of gates, test requirements for scenario, and long-term or stress testing (acm_, 15:26:41)
    11. idea of multiple gates is to have scenario based features could have multiple quality levels (or intermediate gates - project releases) (acm_, 15:28:22)
    12. trevor_intel what is a non-scenario project? is tool/framework projects. not feature projects, scenario-based (acm_, 15:30:24)
    13. feature meet OPNFV requirments and feature-specific requirments (gates) (acm_, 15:31:17)
    14. tool/framework define their own requirments, make them public. (acm_, 15:31:48)
    15. need a set of OPNFV requirments - common across all scenarios AND feature-specific test requirements would need to leverage an existing test framework (acm_, 15:34:09)
    16. - this is where Test WG contributes - making sure existing framworks can support the feature-specific tests. (acm_, 15:34:55)
    17. Ideally, ability to creation of a new test would clearly land in one or two test frameworks - feature project contributes that test. (acm_, 15:37:24)
    18. https://etherpad.opnfv.org/p/RPWG_-_Release_Process (mj_rex, 15:37:51)
    19. there has been some investigation of this, Gabriel (acm_, 15:37:56)
    20. trevor_intel Testing WG is not a development group - more about coordination between test projects - implementation done elsewhere... (acm_, 15:39:38)
    21. trevor_intel someone has to contribute to the test framework projects, for a feature-specific test case. (acm_, 15:40:45)
    22. trevor_intel the way it has worked up till now is that Functest or Yardstick have created the feature-specific tests (acm_, 15:41:34)
    23. trevor_intel the practical reality is that you need to be a Knowledgeable developer of Functest or Yardstick to add feature-specific test cases (acm_, 15:42:53)
    24. dmcbride freature projects have a lot of tests - but these need to be re-orgainzed under this gate structure. (acm_, 15:44:12)
    25. dmcbride REALLY looking for a new Gate Design - a structure to objectively determine promotion for scenarios with a specific feature (acm_, 15:45:40)
    26. how is that different from today? Functest has many tests (acm_, 15:46:26)
    27. georgk we don't follow the small test gate, then get promoted to bigger tests and more meaningful outcome - but this is XCI and we're not there yet (acm_, 15:47:53)
    28. georgk what do we want to change to improve our quality of delivery? (acm_, 15:48:22)
    29. dmcbride example of 3 gates passing for project release, then ... (acm_, 15:48:56)
    30. trevor_intel but .. projects decide these tests. (acm_, 15:49:24)
    31. example gate 1 is passed if success in smoke test and runs on baremetal -- gate 2 is defined by the feature project (acm_, 15:50:33)
    32. trevor - sounds more like the transition from a project release, adding tests for a platform or OPNFV release (acm_, 15:51:20)
    33. trevor_intel question for georgk - do we have the machinery in place to do promotion-levels and gating? (acm_, 15:52:18)
    34. georgk - it was shown by Fatih a year ago, as part of XCI, what would be included. (acm_, 15:53:17)
    35. trevor_intel who will do this??? XCI is very small, 2 people, (acm_, 15:54:09)
    36. dmcbride then we need to put more emphasis on XCI somehow -treat this as a problem to solve (acm_, 15:54:51)
    37. dmcbride APEX PTL is stepping down, has been our most popular installer, in order for OPNFV to move forward, may need to embrace XCI (acm_, 15:56:03)
    38. dmcbride need to recruit developers for XCI trevor_intel <<< this is the place to start! (acm_, 15:56:48)
    39. georgk - so, now we need a group that considers and gives opinions on test cases, dmcbride with gate design, to keep this alive (acm_, 15:58:22)
    40. dmcbride and Fatih will be at OSLS - possibility to discuss (acm_, 15:59:28)
    41. Akraino testing discussed next time! (acm_, 16:00:34)
    42. FYI Akraino Validation Framework proposal https://wiki.akraino.org/display/AK/Akraino+Blueprint+Validation+Framework (trevor_intel, 16:00:39)
    43. homework is to look at the Akraino for next week (acm_, 16:03:01)
    44. David - you need to end the meeting! (acm_, 16:03:36)
    45. Hi Morgan, our meeting has migrated to 1500 UTC, we're done for today, but having trouble ending the meeting for collabot... (acm_, 16:22:58)


Meeting ended at 16:35:05 UTC (full logs).

Action items

  1. (none)


People present (lines said)

  1. acm_ (51)
  2. dmcbride (6)
  3. collabot` (5)
  4. trevor_intel (3)
  5. mj_rex (2)
  6. georgk (1)
  7. acm (0)


Generated by MeetBot 0.1.4.