14:02:54 <rpaik> #startmeeting Dovetail weekly call
14:02:54 <collabot`> Meeting started Fri Mar 11 14:02:54 2016 UTC.  The chair is rpaik. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:02:54 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:02:54 <collabot`> The meeting name has been set to 'dovetail_weekly_call'
14:03:11 <rpaik> #info Ray Paik
14:03:12 <morgan_orange> #info Morgan Richomme
14:04:26 <bryan_att> #info Bryan SULLIVAN
14:04:56 <bryan_att> Ray you are taking minutes?
14:07:04 <bin_> #info Bin Hu
14:07:30 <rpaik> #info dneary raised the question of target audience for the Plugfest
14:07:42 <arturt_> #info Artur Tyloch
14:07:56 <bryan_att> The audience is opnfv. The invitees are software vendors/integrators of OPNFV platform components.
14:08:43 <lylavoie> I agree Brian, this is the vision we crafted for the plugfest: "Provide an event / opportunity for participants to test and demonstrate interoperability of NFV solutions highlighting the OPNFV platform , in conjunction with the OPNFV technical community.  Participation is open to all companies, with membership in OPNFV not required for this event.
14:08:43 <lylavoie> "
14:09:19 <lylavoie> With a set of measured metrics for our success: "Number of participant, Number of test cases performed by participants, Input into OPNFV future releases"
14:09:58 <anac1> #info Ana Cunha
14:10:15 <bryan_att> Don't forget pass/fail rate as a basic metric!
14:11:45 <bryan_att> I take "around" to mean within the scope of the platform.
14:12:07 <bryan_att> Primarily focused on software.
14:14:18 <rpaik> #info lylavoie notes that the intention is bring both hardware/software people together for the Plugfest
14:15:16 <bin_> We avoid "pass/fail" intentionally, because the target is participation, and the concern of "pass/fail" may give unnecessary hump for vendors to consider to participate.
14:15:46 <bin_> If there is no "pass/fail" result, vendors may be more willing to participate
14:15:58 <bryan_att> Davis point was about there not being fully defined NBIs for lifecycle mgmt yet... Thus that's useful but not definitive yet, enough to be viewed as conformance testing.
14:16:07 <rpaik> #info lylavoie notes that we’re not doing certification/conformance type testing
14:16:28 <bryan_att> s/Davis/Dave's/
14:19:30 <rpaik> #info dneary asks what if a participant wants to bring a solution that is not OpensStack-Liberty or ODL-Be
14:23:23 <bryan_att> ok, the integration of additional components within a defined OPNFV scenarios (e.g. nosdn) is a grey area I agree, but adding a new SDNC changes the scenario and risks diluting the OPNFV focus of the plugfest.
14:24:03 <bryan_att> if more vendors want to bring their solutions to the set of options in OPNFV, they are welcome to join and propose projects.\
14:25:05 <rpaik> #info registration will open on Monday (March 14th) so we could get a profile of who would be interested in attending
14:25:24 <bryan_att> overall though the focus should be on the defined OPNFV scope, including the specific upstream *open *source* components that OPNFV is built around, and as needed we can open it to more *open source* components. But not proprietary components.
14:26:59 <bryan_att> re "pass/fail", we need real metrics on the technical success of the plugfest; found bugs are golden - vendors should not fear them because they learn from them; this is not a PR event but a real working event for OPNFV.
14:27:16 <lylavoie> #info Action Item: Morgan Richomme and Dave Neary will help to create some potential scenarios for testing during the plugfest, calling out specific hardware / software / etc. that could be attempted during the week
14:27:48 <bryan_att> #info Bryan will help also to define the scenarios
14:27:55 <lylavoie> Thank you Bryan!
14:28:39 <dneary> There's a lag of some sort
14:29:11 <sajeev> Is there any entry/exit criteria for Vendor/Solutions participating in OPNFV plug fest?
14:29:19 <dneary> lylavoie, bryan_att: I'd like to send those to the C&C Committee for review.
14:29:41 <bryan_att> sure, I'd like them to be part of the minutes
14:29:54 <rpaik> #info lylavoie asks if we can cherrypick from existing test projects for the plugfest
14:30:52 <bryan_att> we do need to establish a baseline expectation so that we ensure the testing is based upon OPNFV scope
14:31:17 <anac1> #link http://artifacts.opnfv.org/yardstick/brahmaputra/docs/userguide/03-list-of-tcs.html (list of yardstick test cases)
14:31:44 <lylavoie> dneary, I think C&C's role is less technical and more in the definition of programs run by OPNFV
14:31:48 <bryan_att> vendors should agree to focus first on the test baseline that we establish; extra time can be spent on adhoc testing or integration work, but the first result should be more info on how OPNFV tests can be supported.
14:33:47 <lylavoie> #info Action Item: Ana Cunha will take the lead in development the list of base test cases for the plugfest from Functest and Yardstick.  These will be the set of common / recommended tests for execution during the plugfest.
14:35:56 <bryan_att> vendors should have experience with the installers they depend upon if any; there should me minimal expectation of support from the installer teams.
14:36:11 <rpaik> #info there is an expectation to have representatives from the install teams participate in the plugfest
14:36:27 <lylavoie> dneary, I think C&C's work will also pick up quite a bit with the definition of the certification program (not related to the plugfest).  The most critical part of that work with the program document, which defines all of the business requirements within the program (i.e. OPNFV membership requirements, logo usage requirements, escalation procedures, program maintenance procedures, etc.)
14:37:12 <dneary> lylavoie, The role of the C&C is to define criteria (for board approval) for a trademark program, potentially a compliance test suite, recommendations for a compliance program
14:37:26 <bryan_att> it's likely that some people with some experience will be there, just as some people with functest/yardstick experience will be there, and that there may be some need of the testers for support; but this should largely be self-supported through the testers being adequately prepared.
14:37:30 <dneary> lylavoie, They may be less technical, but they own, IMHO, the scope of the Plugfest
14:39:10 <bryan_att> by "scenarios" we mean the ones defined for B release? e.g. selection of components/features.
14:40:27 <lylavoie> Are we anticipating participants test running / using a different installer from the one they designed for?
14:40:28 <bryan_att> e.g. a nosdn scenario can be tested by a  vendor/implementer of an OpenStack Libertybased  product/distro
14:41:06 <lylavoie> i.e. changing from Apex to Compass?
14:41:41 <bryan_att> lylavoie: i would not expect so, unless they already had some experience attempting/succeeding at that
14:42:34 <lylavoie> bryan, that was my thought too, so I think the installer team would more be for bugs with specific participants (which the may have found while prepping for the plugfest0
14:42:51 <bryan_att> let's agree today on the scenarios being those in B release scope
14:43:18 <bryan_att> and to use the term "scenarios" consistently in this effort
14:50:55 <bryan_att> so it sounds like consensus on that the baseline set of tests are those supported for the B release scenarios; can we explicitly agree on that? (of course we will need to narrow test scope even in that focus)\
14:51:51 <morgan_orange> I initiated something at the end of the etherpad https://etherpad.opnfv.org/p/OPNFV_Plugfest2016
14:52:35 <lylavoie> bryan, yes, we need to narrow them up a bit, so it doesn't take too long to run
14:53:06 <lylavoie> and we need to ensure there is some aim towards the opportunity to work together, i.e. do more than we can be done by Pharos along
14:53:09 <lylavoie> *alone
14:55:23 <rpaik> #agree that the baseline set of tests for the Plugfest are subset of tests for the B release scenario
14:57:05 <bryan_att> As Frank suggested, we need to clarify what these support teams from the installers will be doing; IMO it's *only* being there to help testers get beyond issues with the installers/tests; of course the vendors should be prepared, but they may need some help during the plugfest.
15:00:24 <bryan_att> Based upon Tetsuya's comments, I think the "Functional Areas" table needs to include a team that will define the hardware expectations/intent of testers. There doesn't seem to be any item in the table covering this at least clearly;
15:02:25 <rpaik> #info discussion on the need for the list of participants (e.g. hardware vendors, VNF vendors, etc.) and what “scenarios" they want to test
15:03:37 <rpaik> #info plugfest breakout session during the Hackfest on Tunesday (15th) at 11am
15:07:04 <rpaik> #endmeeting