14:00:05 <morgan_orange> #startmeeting Test&performance weekly meeting May 5th
14:00:05 <collabot`> Meeting started Thu May 12 14:00:05 2016 UTC.  The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:05 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:00:05 <collabot`> The meeting name has been set to 'test_performance_weekly_meeting_may_5th'
14:00:13 <morgan_orange> #topic call roll
14:00:17 <morgan_orange> #info Morgan Richomme
14:01:52 <morgan_orange> #chair mbeierl jose_lausuch QiLiang
14:01:52 <collabot`> Current chairs: QiLiang jose_lausuch mbeierl morgan_orange
14:02:02 <jose_lausuch> #info Jose Lausuch
14:02:09 <jose_lausuch> can't hear anything though
14:02:36 <ql> #info qiliang
14:02:45 <frankbrockners> #info Frank Brockners
14:02:54 <zhifeng> #info zhifeng
14:03:10 <mbeierl> #info Mark Beierl
14:05:27 <morgan_orange> #info topic Test Criteria
14:05:52 <mtahhan> #info Maryam Tahhan
14:07:52 * frankbrockners wonders what installer #5 would be... Any thoughts?
14:08:05 <mbeierl> Daisy for NFV ?
14:08:22 <mbeierl> A container focussed installed iirc
14:09:53 <mbeierl> https://wiki.opnfv.org/display/PROJ/Daisy+Nfv
14:10:02 <jose_lausuch> #info release criteria applies to scenarios individually
14:10:26 <jose_lausuch> #info as the test group, we have the mission to validate a release and each of the scenarios
14:10:49 <mtahhan> validate and benchmark? :D or just validate
14:11:04 <jose_lausuch> good point!, both
14:11:35 <jose_lausuch> if we had KPI definitions we could also say "validate" when benchmarking :)
14:11:42 <mtahhan> we could
14:13:24 <jose_lausuch> #info the number of features might increase significantly, since the idea is to test everything, this will cause time concerns
14:14:10 <jose_lausuch> #info to validate a scenario (functest case): 4 consecutive runs + smoke tests passing + scenario specific test  (~1h testing time)
14:17:48 <mbeierl> #info Frank suggests the benefit of pulling the definitions of scenarios out of Jenkins and into scenario definition files that scenario owners can manage
14:19:45 <mbeierl> #info Suggestion understood, but probably not doable in C time
14:26:23 <mbeierl> #link https://wiki.opnfv.org/display/SWREL/Test+Release+Criteria
14:32:38 <frankbrockners> #link https://wiki.opnfv.org/display/SWREL/Colorado+scenario+inventory+and+dependencies - List of scenarios
14:34:17 <morgan_orange> #action morgan_orange review Colorado scenario and conbtact scenario owner to list scenario specific test list
14:34:26 <morgan_orange> #topic roadmap
14:34:38 <morgan_orange> #info more resources allocation roadmap
14:38:04 <morgan_orange> #info to be sync with david karr (action from Infra group to setup CI Intra resources)
14:39:54 <morgan_orange> #topic plugfest
14:40:24 <morgan_orange> #info presentations on Functest, Yardstick, StorPerf well received
14:41:36 <morgan_orange> #info several integrations done, Yardstick and Functest launched successfully
14:42:02 <morgan_orange> #info all the installers represented
14:42:45 <morgan_orange> #info first tests collected
14:42:47 <morgan_orange> #link http://plugfest.opnfv.org/testapi/results
14:47:29 <morgan_orange> #topic Approach to testing
14:47:49 <morgan_orange> #info consolidation on OPNFV testing in progress
14:48:04 <morgan_orange> #info one goal = create a white paper to clarify what is testing in OPNFV
14:49:11 <morgan_orange> #info some wiki pages have to be updated
14:49:34 <morgan_orange> #action morgan_orange review testing main page and re-share first attempt on domain coverage
14:50:00 <morgan_orange> #undo
14:50:00 <collabot`> Removing item from minutes: <MeetBot.ircmeeting.items.Action object at 0x2eff810>
14:50:09 <morgan_orange> #action all review testing main page and re-share first attempt on domain coverage
14:50:39 <morgan_orange> #link https://wiki.opnfv.org/display/functest/Testing
14:50:43 <mbeierl> #info Clarification on the test results: The above link shows only results of OPNFV test projects.  It is simple pass/fail and basic information on the criteria.  This database does *not* contain any propriety test results such as VNF performance, etc.
14:51:02 <morgan_orange> #link https://wiki.opnfv.org/display/functest/Testing+Coverage
14:52:13 <morgan_orange> #info presentation of Alan's Draft on Testing approach
15:04:23 <morgan_orange> #stopmeeting
15:04:32 <morgan_orange> #endmeeting