15:03:48 <dmcbride> #startmeeting OPNFV Test Working Group
15:03:48 <collabot`> Meeting started Thu Feb 21 15:03:48 2019 UTC.  The chair is dmcbride. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:03:48 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:03:48 <collabot`> The meeting name has been set to 'opnfv_test_working_group'
15:04:00 <dmcbride> #chair acm
15:04:00 <collabot`> Current chairs: acm dmcbride
15:04:05 <acm_> #info Al Morton
15:04:12 <dmcbride> #info David McBride
15:04:16 <trevor_intel> #info Trevor Cooper
15:08:48 <acm_> #info
15:09:04 <acm_> #info Al Morton
15:09:29 <georgk> hi!
15:09:51 <mj_rex> #info Rex Lee
15:10:21 <acm_> #info add Akraino testing to the agenda
15:11:11 <dmcbride> #chair georgk
15:11:11 <collabot`> Current chairs: acm dmcbride georgk
15:11:57 <dmcbride> #topic 2019 working group planning
15:14:16 <acm_> #agree biweekly meetings and no longer a need for APAC time
15:15:02 <acm_> #topic release gate requirements
15:17:41 <acm_> #info good turn-out today, 9 participants
15:18:49 <acm_> #info dmcbride keep meeting - this group is of vital importance to OPNFV
15:19:55 <acm_> #info Board input - two asks: OVP and a common NFVI infrastructure
15:21:12 <acm_> #info Bin elab on NFVI point: many variations of infrastructure with many scenarios - does not assure interop for service provider customers
15:21:57 <acm_> #info dmcbride a clear Board priorty is Testing!
15:22:57 <acm_> #info dmcbride trying to overhaul rel process - meetings at plugfest were productive, new ideas for process
15:23:47 <acm_> #info dmcbride one idea - specific gates , non-feature scearios define their own (single) gate, for example
15:24:33 <acm_> #info dmcbride this supports more continuous release (above)
15:25:40 <acm_> #info dmcbride for feature releases - have as many as 3 or 4 gates - Trevor Bramwell proposed that TSC takes Recommendation from Testing WG for gate design
15:26:41 <acm_> #info Rec includes: number of gates, test requirements for scenario, and long-term or stress testing
15:28:22 <acm_> #info idea of multiple gates is to have scenario based features could have multiple quality levels (or intermediate gates - project releases)
15:30:24 <acm_> #info trevor_intel what is a non-scenario project? is tool/framework projects. not  feature projects, scenario-based
15:31:17 <acm_> #info feature meet OPNFV requirments and feature-specific requirments (gates)
15:31:48 <acm_> #info tool/framework define their own requirments, make them public.
15:34:09 <acm_> #info need a set of OPNFV requirments - common across all scenarios AND feature-specific test requirements would need to leverage an existing test framework
15:34:55 <acm_> #info - this is where Test WG contributes - making sure existing framworks can support the feature-specific tests.
15:37:24 <acm_> #info Ideally, ability to creation of a new test would clearly land in one or two test frameworks - feature project contributes that test.
15:37:51 <mj_rex> #link https://etherpad.opnfv.org/p/RPWG_-_Release_Process
15:37:56 <acm_> #info there has been some investigation of this, Gabriel
15:39:38 <acm_> #info trevor_intel  Testing WG is not a development group - more about coordination between test projects - implementation done elsewhere...
15:40:45 <acm_> #info trevor_intel someone has to contribute to the test framework projects, for a feature-specific test case.
15:41:34 <acm_> #info trevor_intel  the way it has worked up till now is that Functest or Yardstick have created the feature-specific tests
15:42:53 <acm_> #info trevor_intel  the practical reality is that you need to be a Knowledgeable developer of Functest or Yardstick to add feature-specific test cases
15:44:12 <acm_> #info dmcbride freature projects have a lot of tests - but these need to be re-orgainzed under this gate structure.
15:45:40 <acm_> #info dmcbride REALLY looking for a new Gate Design - a structure to objectively determine promotion for scenarios with a specific feature
15:46:26 <acm_> #info how is that different from today? Functest has many tests
15:47:53 <acm_> #info georgk  we don't follow the small test gate, then get promoted to bigger tests and more meaningful outcome - but this is XCI and we're not there yet
15:48:22 <acm_> #info georgk  what do we want to change to improve our quality of delivery?
15:48:56 <acm_> #info dmcbride example of 3 gates passing for project release, then ...
15:49:24 <acm_> #info trevor_intel  but .. projects decide these tests.
15:50:33 <acm_> #info  example gate 1 is passed if success in smoke test and runs on baremetal  -- gate 2 is defined by the feature project
15:51:20 <acm_> #info trevor - sounds more like the transition from a project release, adding tests for a platform or OPNFV release
15:52:18 <acm_> #info trevor_intel question for georgk - do we have the machinery in place to do promotion-levels and gating?
15:53:17 <acm_> #info georgk - it was shown by Fatih a year ago, as part of XCI, what would be included.
15:54:09 <acm_> #info trevor_intel  who will do this??? XCI is very small, 2 people,
15:54:51 <acm_> #info dmcbride then we need to put more emphasis on XCI somehow -treat this as a problem to solve
15:56:03 <acm_> #info  dmcbride APEX PTL is stepping down, has been our most popular installer,  in order for OPNFV to move forward, may need to embrace XCI
15:56:48 <acm_> #info  dmcbride need to recruit developers for XCI  trevor_intel <<< this is the place to start!
15:58:22 <acm_> #info georgk - so, now we need a group that considers and gives opinions on test cases, dmcbride with gate design, to keep this alive
15:59:22 <trevor_intel> FYI Akraino Validation Framework  proposal https://wiki.akraino.org/display/AK/Akraino+Blueprint+Validation+Framework
15:59:28 <acm_> #info dmcbride and Fatih will be at OSLS - possibility to discuss
16:00:34 <acm_> #info Akraino testing discussed next time!
16:00:39 <trevor_intel> #info FYI Akraino Validation Framework  proposal https://wiki.akraino.org/display/AK/Akraino+Blueprint+Validation+Framework
16:03:01 <acm_> #info homework is to look at the Akraino for next week
16:03:05 <acm_> #endmeeting
16:03:36 <acm_> #info David - you need to end the meeting!
16:03:46 <acm_> #endmeeting
16:22:58 <acm_> #info  Hi Morgan, our meeting has migrated to 1500 UTC, we're done for today, but having trouble ending the meeting for collabot...
16:35:05 <dmcbride> #endmeeting