15:03:48 #startmeeting OPNFV Test Working Group 15:03:48 Meeting started Thu Feb 21 15:03:48 2019 UTC. The chair is dmcbride. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:03:48 Useful Commands: #action #agreed #help #info #idea #link #topic. 15:03:48 The meeting name has been set to 'opnfv_test_working_group' 15:04:00 #chair acm 15:04:00 Current chairs: acm dmcbride 15:04:05 #info Al Morton 15:04:12 #info David McBride 15:04:16 #info Trevor Cooper 15:08:48 #info 15:09:04 #info Al Morton 15:09:29 hi! 15:09:51 #info Rex Lee 15:10:21 #info add Akraino testing to the agenda 15:11:11 #chair georgk 15:11:11 Current chairs: acm dmcbride georgk 15:11:57 #topic 2019 working group planning 15:14:16 #agree biweekly meetings and no longer a need for APAC time 15:15:02 #topic release gate requirements 15:17:41 #info good turn-out today, 9 participants 15:18:49 #info dmcbride keep meeting - this group is of vital importance to OPNFV 15:19:55 #info Board input - two asks: OVP and a common NFVI infrastructure 15:21:12 #info Bin elab on NFVI point: many variations of infrastructure with many scenarios - does not assure interop for service provider customers 15:21:57 #info dmcbride a clear Board priorty is Testing! 15:22:57 #info dmcbride trying to overhaul rel process - meetings at plugfest were productive, new ideas for process 15:23:47 #info dmcbride one idea - specific gates , non-feature scearios define their own (single) gate, for example 15:24:33 #info dmcbride this supports more continuous release (above) 15:25:40 #info dmcbride for feature releases - have as many as 3 or 4 gates - Trevor Bramwell proposed that TSC takes Recommendation from Testing WG for gate design 15:26:41 #info Rec includes: number of gates, test requirements for scenario, and long-term or stress testing 15:28:22 #info idea of multiple gates is to have scenario based features could have multiple quality levels (or intermediate gates - project releases) 15:30:24 #info trevor_intel what is a non-scenario project? is tool/framework projects. not feature projects, scenario-based 15:31:17 #info feature meet OPNFV requirments and feature-specific requirments (gates) 15:31:48 #info tool/framework define their own requirments, make them public. 15:34:09 #info need a set of OPNFV requirments - common across all scenarios AND feature-specific test requirements would need to leverage an existing test framework 15:34:55 #info - this is where Test WG contributes - making sure existing framworks can support the feature-specific tests. 15:37:24 #info Ideally, ability to creation of a new test would clearly land in one or two test frameworks - feature project contributes that test. 15:37:51 #link https://etherpad.opnfv.org/p/RPWG_-_Release_Process 15:37:56 #info there has been some investigation of this, Gabriel 15:39:38 #info trevor_intel Testing WG is not a development group - more about coordination between test projects - implementation done elsewhere... 15:40:45 #info trevor_intel someone has to contribute to the test framework projects, for a feature-specific test case. 15:41:34 #info trevor_intel the way it has worked up till now is that Functest or Yardstick have created the feature-specific tests 15:42:53 #info trevor_intel the practical reality is that you need to be a Knowledgeable developer of Functest or Yardstick to add feature-specific test cases 15:44:12 #info dmcbride freature projects have a lot of tests - but these need to be re-orgainzed under this gate structure. 15:45:40 #info dmcbride REALLY looking for a new Gate Design - a structure to objectively determine promotion for scenarios with a specific feature 15:46:26 #info how is that different from today? Functest has many tests 15:47:53 #info georgk we don't follow the small test gate, then get promoted to bigger tests and more meaningful outcome - but this is XCI and we're not there yet 15:48:22 #info georgk what do we want to change to improve our quality of delivery? 15:48:56 #info dmcbride example of 3 gates passing for project release, then ... 15:49:24 #info trevor_intel but .. projects decide these tests. 15:50:33 #info example gate 1 is passed if success in smoke test and runs on baremetal -- gate 2 is defined by the feature project 15:51:20 #info trevor - sounds more like the transition from a project release, adding tests for a platform or OPNFV release 15:52:18 #info trevor_intel question for georgk - do we have the machinery in place to do promotion-levels and gating? 15:53:17 #info georgk - it was shown by Fatih a year ago, as part of XCI, what would be included. 15:54:09 #info trevor_intel who will do this??? XCI is very small, 2 people, 15:54:51 #info dmcbride then we need to put more emphasis on XCI somehow -treat this as a problem to solve 15:56:03 #info dmcbride APEX PTL is stepping down, has been our most popular installer, in order for OPNFV to move forward, may need to embrace XCI 15:56:48 #info dmcbride need to recruit developers for XCI trevor_intel <<< this is the place to start! 15:58:22 #info georgk - so, now we need a group that considers and gives opinions on test cases, dmcbride with gate design, to keep this alive 15:59:22 FYI Akraino Validation Framework proposal https://wiki.akraino.org/display/AK/Akraino+Blueprint+Validation+Framework 15:59:28 #info dmcbride and Fatih will be at OSLS - possibility to discuss 16:00:34 #info Akraino testing discussed next time! 16:00:39 #info FYI Akraino Validation Framework proposal https://wiki.akraino.org/display/AK/Akraino+Blueprint+Validation+Framework 16:03:01 #info homework is to look at the Akraino for next week 16:03:05 #endmeeting 16:03:36 #info David - you need to end the meeting! 16:03:46 #endmeeting 16:22:58 #info Hi Morgan, our meeting has migrated to 1500 UTC, we're done for today, but having trouble ending the meeting for collabot... 16:35:05 #endmeeting