14:02:02 <trevor_intel1> #startmeeting OPNFV Performance and Test
14:02:02 <collabot`> Meeting started Thu Apr 14 14:02:02 2016 UTC.  The chair is trevor_intel1. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:02:02 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:02:02 <collabot`> The meeting name has been set to 'opnfv_performance_and_test'
14:02:22 <trevor_intel1> #info Trevor Cooper
14:03:25 <YaoguangWang__> #info YaoguangWang
14:05:01 <trevor_intel1> #topic Release Criteria discussion
14:06:06 <mbeierl> #info Mark Beierl
14:06:51 <mtahhan> #info Maryam Tahhan (vsperf)
14:07:34 <anac2> #info Ana Cunha
14:10:15 <trevor_intel1> #info proposal is to make release the same as with release B
14:18:57 <mtahhan> #info we don't want to run daily tests that tie up the whole testing pipeline - resources are a concern to include in the CI pipeline
14:19:11 <mtahhan> #info weekly tests can run for longer
14:19:32 <mtahhan> #info we have to be careful about what tests get included as part of Yardstick and functest ci
14:19:58 <mtahhan> #info functest looking at reducing the test execution time
14:23:07 <mtahhan> #info daily runs should be a subset of test cases that we have run on CI PODs
14:23:41 <mtahhan> #info other/extended test cases run on separate (non CI) PODs
14:26:15 <mbeierl> #info Suggestion: longer running tests run by the individual projects on their own CI pipeline or POD, and then report to a common dashboard
14:29:58 <mtahhan> #info we need to replicate our Results DB
14:30:08 <mtahhan> #info at the moment there's only one
14:31:34 <mtahhan> #info we should investigate and pick what results solution will fit the testing projects needs
14:33:32 <mbeierl> #info general agreement was reached that there should be a single common place for storing results, but how to display and report may be up to different projects
14:37:14 <mtahhan> #info test projects to document all test run times
14:37:56 <mtahhan> #info dedicated PODs for the extended tests are required
14:41:36 <trevor_intel1> #info https://wiki.opnfv.org/display/SWREL/Colorado+Testing+-+Discussion+and+Proposals
14:45:40 <trevor_intel1> #info Jose will update the test matrix asking for test cases proposed to include in CI and include estimate of time to run each test case
14:46:37 <YaoguangWang_> Imo, if a test runs very slow in some senarios, maybe it is not a isuue of test itselt. Maybe bottlenecks happen. we should find out the reason, instead of ignoring it.
14:47:24 <jose_lausuch1> https://wiki.opnfv.org/display/SWREL/Colorado+Testing+-+Discussion+and+Proposals#ColoradoTesting-DiscussionandProposals-FeatureProjectMatrix
14:51:00 <anac2> YaoguangWang_: yes, please provide your input to the collection of test cases
14:54:00 <YaoguangWang_> anac2: okay, i will
14:57:29 <bryan_att> jose_lausuch1: while we're here, can you drop a link to where the test reporting API is documented?
15:00:01 <trevor_intel1> #info Trevor ask David for time in Tuesday release meeting on topics of 1) Proposal re test criteria 2) What feature projects need to know re test projects
15:02:36 <jose_lausuch1> bryan_att: https://wiki.opnfv.org/display/functest/Collection+Of+Test+Results
15:02:49 <trevor_intel1> #info Use Colorado testing page to get started ... Bryan
15:03:07 <trevor_intel1> #endmeeting