15:00:50 <morgan_orange> #startmeeting Testing weekly meeting December 15th
15:00:50 <collabot> Meeting started Thu Dec 15 15:00:50 2016 UTC.  The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:50 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:00:50 <collabot> The meeting name has been set to 'testing_weekly_meeting_december_15th'
15:00:56 <morgan_orange> #topic call role
15:01:01 <morgan_orange> #info Morgan Richomme
15:01:08 <kubi001> #info kubi
15:01:09 <mbeierl> #info Mark Beierl
15:01:24 <trevor_intel> #info Trevor Cooper
15:02:07 <morgan_orange> #info agenda https://wiki.opnfv.org/display/meetings/TestPerf
15:03:34 <morgan_orange> #topic Action point follow up
15:03:44 <morgan_orange> #link http://ircbot.wl.linuxfoundation.org/meetings/opnfv-testperf/2016/opnfv-testperf.2016-12-01-15.00.html
15:04:00 <morgan_orange> #info no action point
15:04:14 <morgan_orange> #topic Feedback from the plugfest
15:04:28 <morgan_orange> #chair trevor_intel mbeierl
15:04:28 <collabot> Current chairs: mbeierl morgan_orange trevor_intel
15:04:44 <morgan_orange> #chair kubi001
15:04:44 <collabot> Current chairs: kubi001 mbeierl morgan_orange trevor_intel
15:05:10 <morgan_orange> #info more people since #1
15:05:19 <jose_lausuch> #info significat improvement: deployments went much better, almost without issues
15:05:34 <morgan_orange> #info installation ready quicker by tuesday most of the installations were completed
15:05:43 <morgan_orange> #chair jose_lausuch
15:05:43 <collabot> Current chairs: jose_lausuch kubi001 mbeierl morgan_orange trevor_intel
15:05:57 <jose_lausuch> #info Titanium installation from WindRiver
15:06:24 <jose_lausuch> #info problems with v3 auth mechanism
15:06:24 <morgan_orange> #info non OPNFV installation challenging but interesting..
15:06:40 <jose_lausuch> #info Storperf managed to get it working with HEAT
15:07:19 <jose_lausuch> #info Functest managed to fix v3 authentication, but Rally failed. JIRAs created and being fixed by Helen
15:08:36 <mbeierl> #link https://gerrit.opnfv.org/gerrit/#/c/25725/12
15:08:52 <mbeierl> #info the proposal from Functest to change authentication to v3
15:09:06 <morgan_orange> #info no so much tests on Colorado...wotk with Titatnium due to Newton so more a hackfest than a plugfest...
15:10:31 <jose_lausuch> #info Results collected to the DB. Issues indentified and posted in the internal plugfest wiki
15:10:40 <morgan_orange> #info more remote PODs for the plugfest, easier than shipping
15:10:43 <mbeierl> #info plugfest had more remote hardware this time, more VPN and remote POD access.  From Intel it worked well, easier than shipping pods
15:11:24 <morgan_orange> #info lots of discussions/meetings (CVP, board, Release, ....)
15:11:42 <mbeierl> ... it was almost too busy :(
15:11:49 <jose_lausuch> #info a lot of focus on Infra meetings (pod descriptor, ci evolution...)
15:13:02 <jose_lausuch> #info use Dovetail proposal about test descriptor file
15:14:29 <morgan_orange> #info lots of people from Open-O
15:14:44 <mbeierl> #info Interest in Open-O integration in Functest
15:15:49 <morgan_orange> #topic discussion testing landing page
15:16:32 <morgan_orange> #info corresponds to one of the Testing priorities in order to give consistency and give a global view
15:16:53 <morgan_orange> #info used to be done manually on the wiki in Colorado (hard to maintain and reflect the reality)
15:17:10 <morgan_orange> #info proto with functest and yardstick but configurable to welcome other test projects
15:18:30 <morgan_orange> #info several filters (installers/version/test projects/ci-loop)
15:20:44 <morgan_orange> #info question use of Jenins APi to check deployment result?
15:26:22 <mbeierl> #info question if Daisy is going to be advanced/stable enough to include in dashboard?
15:27:54 <jose_lausuch> I think as soon as Daisy is running in CI, it can be included here :)
15:29:40 <jose_lausuch> we can use weather icons...
15:29:59 <mbeierl> Where is the code being stored?
15:30:17 <mbeierl> #info dashboard code is being stored in releng
15:30:29 <morgan_orange> #topic Performance & Stress tests discussion
15:32:46 <morgan_orange> #info question which stress test for Danube
15:32:57 <morgan_orange> #info Bottleneck initiated some work on stress tests
15:33:01 <kubi001> yardstick could configure some testing in parallel now.
15:33:17 <morgan_orange> #info yardstick can configure some testing in // now
15:35:09 <kubi001> trevor_intel: It seems that vsperf could generate lots of types traffic at now?
15:36:12 <morgan_orange> gabriel what is your irc login?
15:36:44 <kubi001> gabriel = yuyang_
15:37:04 <morgan_orange> #action yuyang_ plan a Bottleneck update for the testing community beginning of January
15:37:28 <morgan_orange> #info trevor_intel mentioned that the stress test must be easy to reproduce
15:40:54 <bryan_att> #info we need first to organize at a high level on objectives, there are many types of stress and we need to be sure we are using the same terms
15:41:27 <mbeierl> we also need to decide what different stresses occur simultaneously.
15:44:01 <morgan_orange> #action trevor_intel create an etherpad page on stress test: with short term view (Danube) and longer view (stress test strategy)
15:44:17 <kubi001> sounds good
15:45:13 <bryan_att> #info e.g. resource limits of various types; chaos; multiple operations that might result in some resource conflicts; HA-related stress; .... etc
15:46:37 <morgan_orange> #topic VNF testing (on boarding/ load / stress /)
15:46:40 <bryan_att> #info the key is what we are wanting to achieve and how we will measure it. stress tests are not always binary in terms of success/fail assessment
15:47:27 <bryan_att> #info I have several tests already for VNF deployment in Models and we can use them in this stress test if needed
15:49:55 <bryan_att> #info again, is this testing just for the purpose of ensuring that MANO functions work, or are we using the MANO stuff as a tool to deploy and test VNFs of significantly different types... the latter is where the Models project can also provide help, as we develop models for various types of VNFs that exercise significantly different types of NFVI resources
15:51:50 <bryan_att> #info the Danube goal for Models is to have coverage for all the VNFMs supported in this release, with the same or very similar VNF blueprints, so that may be useful here; e.g. you don't need the full Open-O stack (dozens of docker containers) to drive the system using their NFVO/VNFM
15:52:51 <bryan_att> #info we have even fully defined what VNF onboarding is... that's a discussion still underway in the MANO WG
15:54:34 <bryan_att> #info what you *can* get out of such testing is to know that the NFVO/VNFM is interoperable with the OPNFV system, as it successfully deploys the VNF and tests of the VNF succeed - even if you don't know what APIs the VNFM etc use (e.g. Heat vs Neutron/Nova), you will know that it is compatible
15:54:57 <kubi001> bryan_att:  do the models for VNFs could simulate the load(compute and network) of VNFs?
15:55:04 <morgan_orange> #action morgan_orange cretae a page for vnfOnBoarding for testing
15:55:23 <morgan_orange> #topic Update on test API testcase collection update
15:55:45 <morgan_orange> #link http://testresults.opnfv.org/test/api/v1/projects/functest/cases
15:57:34 <bryan_att> kubi001: yes, e.g. for VES we put load and disrupt the workloads at the same time
15:57:51 <morgan_orange> #info any comments welcome
16:00:36 <morgan_orange> #topic documentation
16:00:47 <morgan_orange> #info we need to create an umbrella project
16:01:09 <morgan_orange> #info do you agree with the umbrella project (starting end of January)
16:01:12 <morgan_orange> #agree
16:01:28 <kubi001> +1
16:01:31 <mbeierl> #agree
16:01:35 <trevor_intel> +1
16:01:35 <kubi001> #agree
16:01:37 <morgan_orange> #agreed
16:01:42 <mbeierl> #agreed
16:01:56 <morgan_orange> #topic AoB
16:02:54 <kubi001> bryan_att: Yardstick have a problem with how to decouple the workload from  VNFs? If we can't get the accurate model of VNFs, we can't give the threshold of the NFVI metric
16:03:18 <morgan_orange> #info weekly meetings canceled until end of Decemeber, see you back in January
16:04:12 <morgan_orange> #endmeeting