07:30:09 <anac1> #startmeeting Yardstick work meeting
07:30:09 <collabot> Meeting started Mon Sep 28 07:30:09 2015 UTC.  The chair is anac1. Information about MeetBot at http://wiki.debian.org/MeetBot.
07:30:09 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
07:30:09 <collabot> The meeting name has been set to 'yardstick_work_meeting'
07:30:21 <anac1> #info Ana Cunha
07:30:29 <qiliang_> #info QiLiang
07:30:30 <Patrick_> #info Patrick
07:30:37 <wuzhihui> info wuzhihui
07:31:26 <anac1> can you see my screen on gotomeeting?
07:31:29 <Patrick_> yes
07:31:30 <qiliang_> yes
07:31:38 <anac1> ok, good
07:32:29 <anac1> #topic milestone information
07:33:03 <anac1> #info milestone  C last friday 25/spetember
07:34:50 <jingwen> #info Jingwen Hou
07:35:07 <anac1> #info milestone d december 1st
07:37:23 <anac1> #info requests summarized in epic in yardstick jira
07:38:53 <anac1> #info requests from sfc, ipv6, vnffg, armband, sdnvpn
07:39:29 <Patrick_> I think we also got the request from HA project?
07:39:34 <anac1> #info work until milestone d - have all test specifications ready
07:39:49 <anac1> #info request also form ha, kvm
07:40:32 <anac1> #info plan is to discuss architectural changes based on requests above
07:40:47 <anac1> comments?
07:41:30 <Patrick_> do we finished all of the test cases from the projects in Milestone D?
07:43:11 <anac1> #info the test cases should be executed during december, the test specification (.rst, plus functionality in framework) should be ready for milestone d
07:44:28 <anac1> patrick: did i answer your question ?
07:44:36 <Patrick_> yes
07:44:39 <anac1> ok
07:45:38 <anac1> #info this meeting today is for all of us to understand the scope
07:46:28 <anac1> #info suggestion: for new requests, put in backlog
07:48:52 <anac1> #info test specification should contain where to perform the test
07:49:11 <qiliang_> what is test specification mean ? the yaml file ?
07:49:26 <anac1> good question
07:49:37 <Patrick_> one question: we know that the performance test cases(TC1~TC14?) maybe is not enough for verity the OPNFV platform. so should we continue investigating the performance tools ?
07:50:18 <anac1> #info test specification is the .rst file: test_case_description_template.rst
07:50:33 <anac1> we need to agree on the template
07:51:14 <anac1> Patrick: sure, we need to continue to investigate, TC1 -> TC14 is the beginning
07:51:24 <Patrick_> ok.
07:51:41 <anac1> #info read ETSI TST001 document
07:51:55 <anac1> #info this document is a methodology for NFVI
07:52:28 <anac1> #info go to ETSI-NFV page to find the document, it a draft, but you can download it
07:52:33 <qiliang_> thanks
07:53:56 <anac1> as Patrcik has pointed out, we continue to the add the investigations to expand the metrics and what we test
07:54:52 <anac1> a question on how to handle new test cases ? how should we prioritize?
07:56:37 <anac1> #info the other projects that have requested our support have also their tasks in jira
07:59:15 <anac1> more comments ?
08:00:32 <anac1> #info hardware is a potential problem - where to verify the features for the release
08:01:55 <anac1> any further questions or is it ok to move to point 2?
08:09:16 <anac1> #info yardstick team will continue developing NFVI test cases
08:09:21 <Jingwen_> one question：Most of our test cases only provide samples, do we need to provide some base test cases to verify the ability of a certain aspect?
08:09:36 <Jingwen_> e.g. provide some base cases to verify the storperf using FIO
08:10:26 <anac1> #info a sample is an example how to the tool e.g. fio
08:11:06 <anac1> #info the test case (the .rst file mentioned above) is the the verification of a metric/characteristic
08:13:54 <anac1> did i answer the question?
08:14:01 <Jingwen_> YES
08:14:04 <Patrick_> I think the mean from Jingwen is that , whether we will provide some test cases(formating with ymal) to test the storage perf using FIO
08:14:17 <anac1> yes, we will
08:15:26 <anac1> #info drafts for two test cases will be pushed to gerrit
08:15:30 <Patrick_> great work.
08:15:50 <anac1> #info tc002 and tc001
08:16:27 <anac1> we can have a review in gerrit once the .rst is pushed
08:16:36 <Patrick_> ok
08:17:23 <anac1> more questions?
08:18:05 <anac1> we move to point 2 in the agenda?
08:18:10 <Patrick_> yes
08:18:24 <anac1> #topic walk thruogh jira
08:22:30 <anac1> #info check dashboard and kanban board for overview of the project
08:24:00 <anac1> #info yardstick-19
08:24:16 <anac1> should we include in R2 or postpone?
08:26:19 <anac2> agree to postpone yardstick-19?
08:27:10 <anac2> #agreed yardstick-19 in R2 for vTC
08:31:48 <anac2> #info yardstick-26
08:31:59 <anac2> comments?
08:32:09 <qiliang_> low priority from my side
08:32:44 <anac2> can we agree to remove from r2?
08:32:51 <Patrick_> agree.
08:33:00 <qiliang_> agree
08:33:13 <anac2> #agreed remove yardstick-26 from r2
08:34:29 <anac2> #info yardstick-27
08:34:33 <anac2> comments?
08:35:07 <qiliang_> still not understand what's this jira mean
08:36:01 <anac2> #info this jira means invoking yardstick several times in paralel against the same installation
08:36:13 <Patrick_> several yardstick instance running?
08:36:26 <anac2> yes
08:38:07 <anac2> agree to remove from r2?
08:38:29 <Patrick_> agree
08:38:30 <Patrick_> ok. maybe we don't need this functions to support current test cases. suggest to remove from r2.
08:38:40 <Jingwen_> agree
08:38:58 <anac2> #agreed remove yardstick-27 from r2
08:40:24 <anac2> #info yardstick-32
08:40:31 <qiliang_> also low priority
08:40:58 <anac2> agree to keep it outside R2?
08:41:05 <qiliang_> agree
08:41:20 <anac2> #agreed yardstick-32 not in R2 scope
08:43:50 <anac2> #info yardstick-35 and yardstick-36 in progress
08:44:28 <anac2> #action anac to check yardstick-39
08:45:40 <anac2> #info yardstick-50
08:46:01 <anac2> #info no info
08:47:55 <anac2> #action anac to check yardstick-50
08:48:23 <anac2> #info yardstick-56
08:48:31 <anac2> #info framework user guide
08:52:48 <anac2> #action anac to add a task for adding/expand docstrings in the scenarios, check if the current ones need update
08:53:34 <anac2> #info yardstick team needs to agree on architecture befor writing the architecture chapter
08:54:27 <anac2> #info yardstick-60, 61,62,63,64
08:54:38 <anac2> anything need to downprioritize
08:55:15 <anac2> or keep all in r2?
08:56:10 <qiliang_> for 62,63 are low priority but i will try to finish it in R2.
08:57:27 <anac2> agree to keep in R2 and downpriritize 62,63?
08:57:31 <Patrick_> agreee
08:57:33 <qiliang_> yes
08:57:51 <anac2> #agreed keep 62,63 in R2 but downprioritize
08:59:15 <anac2> #info yardstick-64
09:00:27 <anac2> does it include user guide for adding data in to the database
09:00:58 <qiliang_> yes, and maybe some use guide to config
09:02:52 <anac2> also the description on how the data is structured in the db?
09:03:07 <qiliang_> yes
09:03:51 <anac2> great, thanks !
09:04:09 <anac2> #info yardstick-64 updated with scope
09:05:25 <anac2> #info yardstick-71
09:06:53 <Patrick_> just the document about the yaml?  not include the scernios
09:07:45 <anac2> Patrick: yes
09:08:01 <anac2> the scenarios should be documented in the framework user guide?
09:09:56 <anac2> #action anac to include runners in the docstrings action
09:10:33 <anac2> #info to recap: how to write test cases with existing scenarios and runners is covered in yardstick-71
09:11:12 <anac2> #info the fremwork user guide covers how to add new scenarios, runners (yardstick-56)
09:12:15 <Patrick_> I have a question: the scenarios has more connection with the framework of more connection with the test cases?
09:15:06 <anac2> Patrick: i think how to add a new scenario and a new runners is in the framework user guide; how to use scenarios and runners in the test case user guide
09:15:26 <Patrick_> I am not sure about this. maybe we can discuss by email.
09:15:40 <anac2> ok, send your opinion via e-mail
09:15:54 <Patrick_> ok. thanks
09:15:55 <anac2> ok?
09:16:00 <anac2> ok.
09:16:12 <anac2> #info yardstick-79
09:17:47 <anac2> #info yardstick-79 edited, assigned to per
09:18:43 <anac2> do you want to have a 5 min break?
09:19:30 <Patrick_> actually, we will have dinner after 15 min
09:19:54 <anac2> ok, we continue
09:21:19 <anac2> #info yardstick-82 assigned to per
09:21:38 <anac2> #info yardstick-84
09:21:39 <Patrick_> maybe we can continue to 11:30, then we will have half an hour break for solving  our hunger :) ?
09:22:39 <anac2> Patrick, ok, we have a 1 hour break
09:22:42 <Patrick_> good:)
09:23:04 <anac2> do you have time to continue after that for 1 hour ?
09:23:07 <Patrick_> yes
09:23:20 <anac2> ok, 7 minutes to go
09:24:02 <anac2> anyone working with juju ? yardstick-84?
09:24:36 <anac2> should we downprioritize yardstick-84?
09:24:39 <qiliang_> agree
09:24:48 <Patrick_> agree
09:24:56 <anac2> keep in R2?
09:26:09 <anac2> #agreed yardstick-84 downprioritize, keep in R2
09:26:39 <Patrick_> maybe we can connect with the Genesis project to talk about the installer support?
09:26:48 <Patrick_> maybe we can connect with the Genesis project to talk about the installer support?
09:27:27 <anac2> Patrick, yes, agree
09:28:00 <anac2> #action connect with Genesis to discuss installer support
09:28:19 <anac2> #info yardstick-86 is closed as opensteak is no longer in R2
09:29:28 <anac2> #info yardstick-88 assigned to per
09:29:37 <anac2> let's have a 1 hour break !
09:29:43 <Patrick_> by the way, should yardstick be create a jira on Genesis for the install recognition? I saw Doctor and Qtip has do the things.
09:30:17 <anac2> Patrick, let's discuss in 1 hour, ok?
09:30:25 <Patrick_> sure, thanks
09:30:29 <anac2> thanks
09:30:50 <Patrick_> see you
10:28:05 <anac2> restart in 2 minutes
10:28:28 <Patrick_> ok:)
10:29:20 <Jingwen_> ok
10:30:07 <anac2> let's continue
10:30:20 <qiliang_> ok
10:30:28 <anac2> we were discussing the installers ?
10:30:57 <anac2> dependencies to the installers?
10:31:06 <Patrick_> yes.
10:33:07 <anac2> i don't see any direct dependencies to the installers, what did you want to ponit out Patrick?
10:33:13 <anac2> point
10:33:18 <Patrick_> for example, the HA solution from Fuel and Compass is different
10:33:42 <anac2> ok, then it is the test case
10:33:51 <Patrick_> yes
10:34:10 <anac2> we need then to go an discuss with genesis, it might be other cases we'll find out later
10:35:22 <anac2> do you think is it genesis or the installers invidually we need to address?
10:35:24 <Patrick_> and we should find out whether other installer have special features to not support the yardstick test cases running.
10:36:19 <anac2> so, let's see, there are dependencies to the installers from the perspective of a test case, but not framework
10:36:36 <qiliang_> did installer should pass all test case yardstick create ?
10:36:44 <Patrick_> and from genesis project, does he ask yardstick team give the requirment about installation?
10:36:51 <Patrick_> https://jira.opnfv.org/browse/GENESIS-53
10:37:19 <Patrick_> not from the qtip projects
10:37:35 <Patrick_> not only from
10:37:51 <Patrick_> some other projects also provide
10:38:17 <Patrick_> copper, promise, doctor, vsperf
10:39:05 <Patrick_> so should we provide the requirement from the yardstick?
10:40:13 <anac2> shouldn't we be responsible for the dependencies? or is it genesis?
10:40:42 <Patrick_> I am not sure.
10:40:49 <anac2> me neither
10:41:16 <anac2> ok, let's do like this, i'll attend the genesis meeting and discuss with them
10:41:21 <Patrick_> so maybe we can connect with genesis
10:41:23 <Patrick_> agree
10:41:47 <anac2> everyone is welcome to join, but perhaps very late for some of you
10:42:21 <anac2> #action anac to join genesis meeting and discuss dependencies from yardstick - yardstick or genesis to solve
10:43:06 <Patrick_> ok
10:43:18 <anac2> ok, i'll inform you all after the genesis meeting today
10:43:26 <Patrick_> thanks
10:43:36 <anac2> thanks for bringing this up
10:44:40 <anac2> continue the backlog
10:45:19 <anac2> #info yardstick-94
10:46:12 <Patrick_> has it already solved?
10:46:31 <anac2> not the test case, the scenario is solved
10:46:43 <Patrick_> ok
10:47:17 <anac2> will keep open, to be taken from backlog when someone is available
10:47:44 <anac2> #info yardstick-99
10:47:52 <anac2> any updates?
10:48:16 <Patrick_> kubi is not here. I think we can down priorities
10:48:42 <anac2> ok, we will keep in R2 but down prioritize
10:48:45 <Patrick_> yes.
10:49:41 <anac2> #agreed yardstick-99: keep in R2 down prioritize
10:50:07 <anac2> #info yardstick-100
10:50:14 <anac2> #info not in R2
10:50:16 <anac2> agree?
10:51:04 <qiliang_> i think that's depend on the usage of external script
10:51:29 <anac2> jnon suggest to add to R2, down prioritize
10:51:35 <Patrick_> I am not sure whether HA scenarios will use the hooks.
10:51:36 <qiliang_> agree
10:52:09 <Patrick_> agree with jnon
10:52:13 <anac2> #agreed yardstick-100 in R2, down prioritize
10:54:09 <anac2> #info yardstick-109
10:56:45 <anac2> meeting with vsperf will be on wednesday, even thuogh many of you will be on vacation
10:56:52 <anac2> we will keep you updated
10:57:32 <anac2> suggested to go back to this discussion after meeting with vsperf and after the chinese national holiday
10:57:39 <Patrick_> ok
10:58:01 <anac2> #info keep yardstick-109 not in R2 until further discussion
10:58:40 <anac2> #info yardstick-113
10:59:32 <anac2> #info lmbench scenario needs to be changed
10:59:58 <anac2> #action anac to write a task for changing lmbench scenario for yardstick-113
11:00:43 <anac2> #info yardstick-114 & 115
11:01:11 <anac2> keep it unassigned until someone can take it?
11:01:20 <qiliang_> ok
11:01:23 <Patrick_> ok.
11:01:39 <anac2> #agreed yardstick-114 and 115 in backlog
11:02:14 <anac2> #info yardstick-123
11:04:05 <anac2> jnon suggests to have in R2
11:04:21 <anac2> a question is where to store the log files
11:04:45 <qiliang_> what need to log ?
11:05:48 <anac2> good question, mainly log files from the services for troubleshooting
11:06:18 <qiliang_> is this requirement mainly for ha ?
11:06:37 <anac2> ha and also general
11:07:08 <anac2> jnon can start a discussion on jira yardstick-123
11:07:23 <qiliang_> great thanks.
11:07:29 <anac2> #action jnon add info/start discussion on yardstick-123
11:08:06 <Patrick_> by the way, I will call the guys from Vaccine first to think about and investigate the test cases about the availability using in yardstick.
11:08:26 <anac2> Patrick: good idea
11:08:50 <anac2> #info yardstick-125 & 126
11:09:05 <anac2> suggest to keep unassigned in the backlog
11:09:28 <Patrick_> ok
11:09:48 <anac2> #agreed yardstick-125 & 126 - keep in backlog
11:10:13 <anac2> #info yardstick-130
11:10:34 <anac2> jnon suggest to keep in R2 downprioritize
11:10:43 <Patrick_> agree.
11:11:06 <anac2> #agreed yardstick-130 keep in backlog, down prioritize
11:11:31 <anac2> #info yardstick-135
11:11:42 <Patrick_> you can assign to me
11:11:57 <anac2> Patrick, I was going to ask exactly this
11:11:58 <Patrick_> :)
11:12:27 <Patrick_> panghao1
11:12:50 <qiliang_> anac2 panghao1
11:13:02 <anac2> thanks ! i was wondering this !
11:14:16 <anac2> #info assigned to Patrick
11:14:32 <anac2> #info yardstick-136
11:14:50 <anac2> #info on-going, jnon's highest prio at the moment
11:15:12 <anac2> #info yardstick-137
11:17:01 <anac2> #info on-going, code will be added
11:17:46 <anac2> #info yardstick-138
11:18:50 <anac2> #info on-track for R2
11:20:07 <anac2> #info yardstick-149
11:20:15 <anac2> #agreed assign to patrick
11:20:22 <Patrick_> ok
11:20:55 <anac2> #info yardstick-150
11:21:14 <anac2> #info can we have a decision for cyclic test (yardstick-122)?
11:22:08 <anac2> qilinag and jnon: agree to move forward ?
11:22:15 <anac2> jnon agrees
11:22:24 <anac2> qiliang: ok ?
11:22:42 <qiliang_> don't what else can i do for kvm
11:23:37 <anac2> ca you implement the solution as described in jörgen's comment is yardstick-122 ?
11:23:41 <anac2> can
11:23:44 <qiliang_> jnon gave to suggestion
11:23:48 <qiliang_> two
11:24:05 <anac2> jnon: which suggestion ?
11:24:38 <anac2> jnon says this part: A quick simple solution would be to allow yardstick to load a yaml file with no context in it (only scenario and runner parts). Then remove the 'host' and 'target' parameters from the cyclictest scenario description and add a couple of new scenario specific options to cyclictest (e.g. ip to test host, external script location, ssh credentials etc.)
11:25:22 <qiliang_> ok, i will move forward.
11:25:34 <anac2> great, thanks !
11:25:58 <anac2> qiliang: add question to jörgen in jira yardstick-122 if it is unclear
11:26:06 <qiliang_> ok
11:27:17 <anac2> #agreed implement solution in yardstick-122 as per comment in jira
11:27:39 <anac2> #info continue discusion in jira if required
11:27:59 <anac2> #info yardstick-151
11:28:43 <anac2> #info anac will contact the sdnvpn project for understanding their test cases, everyone is welcome to join the opnfv meeting
11:29:02 <anac2> #info yardstick-153
11:30:23 <anac2> #info test cases slogans in https://etherpad.opnfv.org/p/testing_requests_for_b_release
11:30:35 <Patrick_> so maybe the wiki about https://wiki.opnfv.org/feature_test_project_matrix%20   should be updated also
11:31:00 <anac2> Patrick: yes, I will update it, thanks !
11:31:15 <anac2> #action anac to update https://wiki.opnfv.org/feature_test_project_matrix%20
11:31:51 <anac2> #action anac will contact the ipv6 project to ask them to start the test specifications
11:32:23 <anac2> #info yardstick-154
11:32:45 <anac2> #info need to have an ARM glance image
11:32:56 <anac2> #info not all test tools can be supported
11:33:18 <anac2> #action anac to add task for investigation
11:33:38 <anac2> #info yardstick-155
11:33:52 <anac2> #info anac to contact nvffg project for details
11:34:08 <anac2> #info idea is to reuse the work for SFC testing
11:34:36 <anac2> #info yardstick-156
11:34:52 <qiliang_> do we know what is sfc's need ?
11:34:53 <anac2> this is about installing the vTC in the opnfv community lab
11:35:22 <anac2> qiliang: i have discussed with sfc, will write an e-mail to mailing lists
11:35:40 <anac2> qiliang: first set is to ping the service functions
11:36:12 <anac2> qiliang: we need to understand how to use their modelling in yardstick
11:36:20 <qiliang_> got it, thanks
11:37:33 <anac2> #action anac to add tasks to sfc jira epic to clarify the work
11:38:31 <anac2> i think we went through the entire backlog, anything missing?
11:39:02 <anac2> we didn't manage the architecture, as i suspected
11:39:15 <anac2> next week, when you are all back?
11:39:48 <anac2> thursday next week work?
11:39:48 <Patrick_> from Oct. 1~7, we have the holiday.
11:40:06 <anac2> october 8th ?
11:40:32 <anac2> UTC 7:30 - 9:30 ?
11:40:47 <Patrick_> ok.
11:40:50 <qiliang_> ok
11:41:02 <Patrick_> it would be ok for me.
11:41:05 <anac2> great, will book
11:41:19 <anac2> in the meantime, continue in the ha etherpad
11:41:35 <Patrick_> in fact , I create a new etherpad
11:41:40 <anac2> link?
11:41:54 <anac2> ok, in the ha etherpad there is a link
11:42:00 <Patrick_> https://etherpad.opnfv.org/p/yardstick_framework
11:42:07 <anac2> great, thanks
11:42:09 <qiliang_> https://etherpad.opnfv.org/p/yardstick_framework
11:42:20 <Patrick_> but I have not finished yet
11:42:30 <anac2> #action all to add comments/suggestion in https://etherpad.opnfv.org/p/yardstick_framework
11:43:01 <anac2> we'll use this for the architecture discussion next thursday oct 8th, UTC 7:30 - 9:30
11:43:02 <Patrick_> great. let the yardstick_ha just talk the test cases about the HA.
11:43:06 <anac2> yes
11:43:46 <anac2> great meeting, thanks all - let's keep in touch, irc, jira, etherpad
11:43:54 <Patrick_> anac, shall we get the yardstick roadmap now?
11:43:54 <anac2> have a good holiday
11:44:04 <anac2> do you have time ?
11:44:45 <Patrick_> hehe:) maybe we can discuss after the holiday.
11:45:13 <anac2> i will create a roadmap and put in the wiki, we can discuss after the holiday ?
11:45:16 <Patrick_> great
11:45:27 <anac2> ok, my head is spinning
11:45:32 <anac2> let's end the meeting
11:45:39 <qiliang_> he he :)
11:45:39 <anac2> thanks for today
11:45:41 <Patrick_> thank you :)
11:45:46 <anac2> #endmeeting