========================================= #opnfv-testperf: OPNFV Test Working Group ========================================= Meeting started by dmcbride at 15:03:48 UTC. The full logs are available at http://ircbot.wl.linuxfoundation.org/meetings/opnfv-testperf/2019/opnfv-testperf.2019-02-21-15.03.log.html . Meeting summary --------------- * Al Morton (acm_, 15:04:05) * David McBride (dmcbride, 15:04:12) * Trevor Cooper (trevor_intel, 15:04:16) * (acm_, 15:08:48) * Al Morton (acm_, 15:09:04) * Rex Lee (mj_rex, 15:09:51) * add Akraino testing to the agenda (acm_, 15:10:21) * 2019 working group planning (dmcbride, 15:11:57) * good turn-out today, 9 participants (acm_, 15:17:41) * dmcbride keep meeting - this group is of vital importance to OPNFV (acm_, 15:18:49) * Board input - two asks: OVP and a common NFVI infrastructure (acm_, 15:19:55) * Bin elab on NFVI point: many variations of infrastructure with many scenarios - does not assure interop for service provider customers (acm_, 15:21:12) * dmcbride a clear Board priorty is Testing! (acm_, 15:21:57) * dmcbride trying to overhaul rel process - meetings at plugfest were productive, new ideas for process (acm_, 15:22:57) * dmcbride one idea - specific gates , non-feature scearios define their own (single) gate, for example (acm_, 15:23:47) * dmcbride this supports more continuous release (above) (acm_, 15:24:33) * dmcbride for feature releases - have as many as 3 or 4 gates - Trevor Bramwell proposed that TSC takes Recommendation from Testing WG for gate design (acm_, 15:25:40) * Rec includes: number of gates, test requirements for scenario, and long-term or stress testing (acm_, 15:26:41) * idea of multiple gates is to have scenario based features could have multiple quality levels (or intermediate gates - project releases) (acm_, 15:28:22) * trevor_intel what is a non-scenario project? is tool/framework projects. not feature projects, scenario-based (acm_, 15:30:24) * feature meet OPNFV requirments and feature-specific requirments (gates) (acm_, 15:31:17) * tool/framework define their own requirments, make them public. (acm_, 15:31:48) * need a set of OPNFV requirments - common across all scenarios AND feature-specific test requirements would need to leverage an existing test framework (acm_, 15:34:09) * - this is where Test WG contributes - making sure existing framworks can support the feature-specific tests. (acm_, 15:34:55) * Ideally, ability to creation of a new test would clearly land in one or two test frameworks - feature project contributes that test. (acm_, 15:37:24) * LINK: https://etherpad.opnfv.org/p/RPWG_-_Release_Process (mj_rex, 15:37:51) * there has been some investigation of this, Gabriel (acm_, 15:37:56) * trevor_intel Testing WG is not a development group - more about coordination between test projects - implementation done elsewhere... (acm_, 15:39:38) * trevor_intel someone has to contribute to the test framework projects, for a feature-specific test case. (acm_, 15:40:45) * trevor_intel the way it has worked up till now is that Functest or Yardstick have created the feature-specific tests (acm_, 15:41:34) * trevor_intel the practical reality is that you need to be a Knowledgeable developer of Functest or Yardstick to add feature-specific test cases (acm_, 15:42:53) * dmcbride freature projects have a lot of tests - but these need to be re-orgainzed under this gate structure. (acm_, 15:44:12) * dmcbride REALLY looking for a new Gate Design - a structure to objectively determine promotion for scenarios with a specific feature (acm_, 15:45:40) * how is that different from today? Functest has many tests (acm_, 15:46:26) * georgk we don't follow the small test gate, then get promoted to bigger tests and more meaningful outcome - but this is XCI and we're not there yet (acm_, 15:47:53) * georgk what do we want to change to improve our quality of delivery? (acm_, 15:48:22) * dmcbride example of 3 gates passing for project release, then ... (acm_, 15:48:56) * trevor_intel but .. projects decide these tests. (acm_, 15:49:24) * example gate 1 is passed if success in smoke test and runs on baremetal -- gate 2 is defined by the feature project (acm_, 15:50:33) * trevor - sounds more like the transition from a project release, adding tests for a platform or OPNFV release (acm_, 15:51:20) * trevor_intel question for georgk - do we have the machinery in place to do promotion-levels and gating? (acm_, 15:52:18) * georgk - it was shown by Fatih a year ago, as part of XCI, what would be included. (acm_, 15:53:17) * trevor_intel who will do this??? XCI is very small, 2 people, (acm_, 15:54:09) * dmcbride then we need to put more emphasis on XCI somehow -treat this as a problem to solve (acm_, 15:54:51) * dmcbride APEX PTL is stepping down, has been our most popular installer, in order for OPNFV to move forward, may need to embrace XCI (acm_, 15:56:03) * dmcbride need to recruit developers for XCI trevor_intel <<< this is the place to start! (acm_, 15:56:48) * georgk - so, now we need a group that considers and gives opinions on test cases, dmcbride with gate design, to keep this alive (acm_, 15:58:22) * dmcbride and Fatih will be at OSLS - possibility to discuss (acm_, 15:59:28) * Akraino testing discussed next time! (acm_, 16:00:34) * FYI Akraino Validation Framework proposal https://wiki.akraino.org/display/AK/Akraino+Blueprint+Validation+Framework (trevor_intel, 16:00:39) * homework is to look at the Akraino for next week (acm_, 16:03:01) * David - you need to end the meeting! (acm_, 16:03:36) * Hi Morgan, our meeting has migrated to 1500 UTC, we're done for today, but having trouble ending the meeting for collabot... (acm_, 16:22:58) Meeting ended at 16:35:05 UTC. People present (lines said) --------------------------- * acm_ (51) * dmcbride (6) * collabot` (5) * trevor_intel (3) * mj_rex (2) * georgk (1) * acm (0) Generated by `MeetBot`_ 0.1.4