08:00:10 <kubi> #startmeeting Yardstick work meeting
08:00:10 <collabot`> Meeting started Thu May 26 08:00:10 2016 UTC.  The chair is kubi. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:10 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:10 <collabot`> The meeting name has been set to 'yardstick_work_meeting'
08:00:16 <kubi> #topic call role
08:00:20 <kubi> #info Kubi
08:00:36 <Mingjiang6> #info Mingjiang
08:00:38 <NokMikeR> #info Michael Rooke, Nokia
08:01:15 <Kanglin> #info Kanglin
08:01:37 <kubi> fuqiao: ping
08:01:39 <YangJian> #info YangJian, CMCC
08:01:56 <MatthewLi> #info Jun Li
08:02:18 <kubi> #topic HA testing
08:02:30 <kubi> we invited HA team to discuss the detail of HA testing.
08:02:38 <fuqiao> #info fuqiao
08:02:52 <kubi> welcome fuqiao and yangjian
08:02:58 <Mingjiang6> welcome
08:03:06 <fuqiao> Having problem with my laptop app, have to use cellphone app
08:03:26 <kubi> fuqiao: that's ok
08:04:13 <kubi> fuqiao: but it may not convenient with etherpad discussion
08:04:23 <Mingjiang6> you may use http://webchat.freenode.net/
08:04:25 <fuqiao> Yes...
08:04:47 <Mingjiang6> web client of irc
08:04:52 <YangJian> i am using https://kiwiirc.com/client
08:05:51 <kubi> we delivered two test cases about HA testing at Rel B.
08:06:13 <Mingjiang6> #link http://artifacts.opnfv.org/yardstick/brahmaputra/docs/userguide/opnfv_yardstick_tc019.html
08:06:20 <fuqiao> Let me try
08:06:58 <qiliang27> #info qiliang
08:07:08 <fuqiao-11> got it. kiwi works~~~
08:07:14 <fuqiao-11> Thank you!
08:07:21 <kubi> fuqiao-11: that's great
08:07:33 <Mingjiang6> tc019 verify the high availability of the service provided by OpenStack (like nova-api, neutro-server) on control node.
08:07:33 <Mingjiang6> It kills the processes of a specific Openstack service on a selected control node, then checks whether the request of the related Openstack command is OK and the killed processes are recovered.
08:07:33 <Mingjiang6> In this test case, an attacker called “kill-process” is provided.
08:07:35 <fuqiao-11> can you send the etherpad link again?
08:08:06 <Mingjiang6> #link http://artifacts.opnfv.org/yardstick/brahmaputra/docs/userguide/opnfv_yardstick_tc019.html
08:08:24 <fuqiao-11> OK
08:09:26 <Mingjiang6> tc025 will verify the high availability of controller node. When one of the controller node abnormally shutdown, the service provided by it should be OK. It shutdowns a specified controller node with some fault injection tools (host-shutdown), then checks whether all services provided by the controller node are OK with some monitor tools.
08:09:35 <Mingjiang6> #link http://artifacts.opnfv.org/yardstick/brahmaputra/docs/userguide/opnfv_yardstick_tc025.html
08:10:08 <kubi> Mingjiang6:  that's a great review of HA testing for Brahmaputra
08:10:51 <kubi> so, at Rel C, we will make our HA testing moving on.
08:11:21 <kubi> fuqiao-11: what's your plan about HA testing in Rel C?
08:11:46 <fuqiao-11> We actually had some talk with Yimin at the beginning of this year
08:12:09 <fuqiao-11> There are several aspects
08:14:41 <kubi> fuqiao-11: are you still here?
08:15:26 <fuqiao-11> sorry, I am also in another meeting...
08:15:30 <fuqiao-11> with our GM....
08:16:26 <kubi> fuqiao-11: that's ok, so could Yangjiang  repersent HA team?
08:18:26 <kubi> i guess we need another slot for HA team.
08:19:08 <kubi> fuqiao-11: please let us know if you come back
08:20:09 <fuqiao-11> 1, the testing in B release only cover limited services
08:20:09 <kubi> #topic HA enhanced framework
08:20:21 <kubi> #undo
08:20:21 <collabot`> Removing item from minutes: <MeetBot.ircmeeting.items.Topic object at 0x2e38a50>
08:20:38 <kubi> #topic HA testing
08:20:56 <fuqiao-11> can we extend the test cases to include more service?
08:21:04 <kubi> yes, we have plan to extend to more services
08:21:09 <Mingjiang6> as for 1, yardstick would extend to more services.
08:21:25 <fuqiao-11> OK
08:21:26 <Mingjiang6> #link https://etherpad.opnfv.org/p/yardstick_ha_tc008
08:21:38 <Mingjiang6> #link https://etherpad.opnfv.org/p/yardstick_ha_tc009
08:21:43 <Mingjiang6> #link https://etherpad.opnfv.org/p/yardstick_ha_tc010
08:21:48 <kubi> #info yardstick would extend to more services.
08:21:53 <Mingjiang6> #link https://etherpad.opnfv.org/p/yardstick_ha_tc011
08:22:00 <Mingjiang6> #link https://etherpad.opnfv.org/p/yardstick_ha_tc012
08:22:30 <fuqiao-11> will yardstick also support SDN controller testing?
08:23:05 <kubi> not so far, it would be better if yardstick could integration with CPerf
08:23:18 <fuqiao-11> also, the testing only cover fuel, how about the other deployment tools?
08:23:50 <Mingjiang6> above is about services 'neutron-server', 'keystone', 'glance-api', 'cinder-api', 'swift-proxy'
08:23:52 <kubi> fuqiao-11: that's a good suggestion. we should extend to more installer
08:24:00 <fuqiao-11> Great
08:24:12 <kubi> Mingjiang6: thank you for your great work
08:24:44 <kubi> #info Mingjiang list more services HA test case about  'neutron-server', 'keystone', 'glance-api', 'cinder-api', 'swift-proxy'
08:25:32 <Mingjiang6> thanks, most of them are collected by kanglin.
08:25:42 <kubi> #info yardstick should extend to more installer to support HA testing. for now, just fuel.
08:25:59 <kubi> Kanglin: Many thanks
08:26:35 <fuqiao-11> also, yimin mentioned about we can imporve the automation of the whole framework
08:27:11 <fuqiao-11> I don't know if this is possible now?
08:27:21 <kubi> fuqiao-11: yes, HA framework enhance has been started
08:28:01 <fuqiao-11> can yardstick provide network failure testing?
08:28:12 <kubi> Qiu Juan and Kanglin will response for this evolution
08:29:05 <Kanglin> fuqiao-11: we have designed another case for network:https://etherpad.opnfv.org/p/yardstick_ha_tc003
08:29:16 <kubi> Kanglin: that's great
08:30:47 <Kanglin> It's the case on network interface failure of the node, and we are considering other network failures
08:31:41 <kubi> fuqiao-11: Qiu Juan introduced HA enhanced framework at previous meeting.  here is what they want to do. https://etherpad.opnfv.org/p/yardstick-ha-new-framework-demo
08:32:15 <kubi> that would make HA framework much more flexible
08:32:25 <fuqiao-11> OK
08:33:02 <kubi> #info yardstick will provide network failure testing https://etherpad.opnfv.org/p/yardstick_ha_tc003
08:33:22 <fuqiao-11> how about storage? do you have storage testing?
08:34:08 <kubi> fuqiao-11: do you mean storage HA testing?
08:34:27 <Kanglin> fuqiao-11: Currently designed a case for disk I/O blocking: https://etherpad.opnfv.org/p/yardstick_ha_tc005
08:34:37 <fuqiao-11> OK
08:34:52 <fuqiao-11> seems you have developed lots of HA testcases.
08:35:09 <fuqiao-11> Can we have a total list for all these HA  testcases
08:35:55 <kubi> now we have list test cases from https://etherpad.opnfv.org/p/yardstick_ha_tc003 to TC012
08:36:32 <fuqiao-11> all on etherpad?
08:36:44 <Kanglin> fuqiao-11: yes
08:36:48 <fuqiao-11> Great
08:36:56 <fuqiao-11> we will look into these in detail
08:37:09 <fuqiao-11> Do you plan to have all these cases in C release
08:37:12 <kubi> fuqiao-11: please feel free to give comments
08:37:46 <kubi> we will give an priority for each test cases.
08:38:25 <kubi> the comments from you team are very important
08:38:46 <fuqiao-11> OK
08:39:09 <fuqiao-11> We may need another discussion after we look into these cases in detail
08:39:25 <kubi> yes, of course.
08:40:05 <kubi> this meeting is a begining of collaboration in Rel C
08:40:14 <fuqiao-11> Great
08:40:57 <kubi> so, I'd like to start other topic if there is no more question.
08:41:08 <fuqiao-11> OK
08:41:14 <fuqiao-11> sorry for low response...
08:41:20 <kubi> thank you fuqiao and Yangjian
08:41:31 <kubi> fuqiao-11: np
08:41:44 <kubi> #topic DPDK test cases
08:42:05 <kubi> #info Akos Farago and zhihui wu are working on netwrok testing with DPDK.
08:42:15 <kubi> zhihui_wu: ping
08:42:22 <zhihui_wu> info zhihui_wu
08:42:37 <kubi> that is a great work, i know there will be a lot of errors and fails on the road.
08:42:50 <kubi> i believe that would be overcome though your great effort and collaboration.
08:43:01 <zhihui_wu> yes, I received the email from Akos.
08:43:14 <kubi> zhihui_wu: that's great
08:43:19 <kubi> One more helpful info is that you could use apt-get to install DPDK in Ubuntu 16.04.
08:43:19 <zhihui_wu> He got much progress on Yardstick-232
08:43:48 <zhihui_wu> I replied the email and cc to Rex.
08:44:16 <Mingjiang6> zhihui_wu: yes, i can receive the email.
08:44:17 <kubi> yes, that's great.
08:44:47 <zhihui_wu> Akos worked on wily. I always worked on trusty before.
08:45:02 <kubi> zhihui_wu: And i knew that someone has successfully setup up DPDK on Ubuntu 16.04.
08:45:07 <kubi> trusty is 14.04
08:45:30 <zhihui_wu> yes. you mean it is better work on 16.04?
08:46:46 <kubi> zhihui_wu: yes, just is suggestion, you can discuss with Akos later, xenial is 16.04.
08:46:54 <zhihui_wu> I think we need discuss about building a new image or modifing the orignal one.
08:47:15 <kubi> yes, that's real thing we need to do
08:49:08 <kubi> due to Akos miss this meeting, the simple way is we give comments on the patch which Akos submited
08:49:32 <kubi> i will invite Akos to attend next IRC meeting.
08:50:26 <zhihui_wu> I just know the installer fuel support hugepage configuration starting from fuel 9.0.
08:50:57 <zhihui_wu> Do the another installers support hugepage? compass,apex,joid
08:51:02 <kubi> yes, i check with compass team, they still not support hugepage
08:52:11 <kubi> DPDK testing is not a generic testing, so we needn't to make it run in all scenario.
08:52:57 <kubi> i will push more installer to support it. but this will take some time
08:53:02 <zhihui_wu> If there is not installer support it, no pod can run DPDK test case.
08:53:37 <zhihui_wu> Yes, I agree with that.
08:53:45 <kubi> is fuel already support it now?
08:53:58 <NokMikeR> Should the DPDK test cases be more generic so they are applicable to e.g ODP and ARMBAND?
08:54:11 <zhihui_wu> The current fuel version in community is 8.0/
08:54:39 <kubi> zhihui_wu: i see. i will check with installer team
08:55:11 <kubi> NokMikeR: could give us more detail infomation about it
08:56:01 <NokMikeR> Will discuss it with the ARMBAND project to make them aware - relevant test cases need drafting.
08:57:40 <kubi> NokMikeR: yes, that's great. it would be better if our test case will be more generic for other projects
08:58:04 <kubi> NokMikeR: please let us know if something need us to do.
08:58:39 <NokMikeR> That would be ideal re generic cases and will liase back from ARMBAND if there is anything for yardstick to do.
08:58:42 <Yaoguang_Wang> As i know, almost half *vsperf* test cases in B-release run with DPDK. Which installer it used can guide us.
08:59:28 <kubi> Yaoguang_Wang: that's a great idea, we can learn something from Vsperf team.
09:00:29 <Yaoguang_Wang> kubi: yep, if we gonna collaborate with vsperf, DPDK issue must be resolved.
09:00:34 <zhihui_wu> AFAIK, the vtc test cases also run with DPDK.
09:01:22 <Yaoguang_Wang> zhihui_wu: Does it require the host or POD to support DPDK?
09:02:01 <zhihui_wu> If we follow the Akos's way, we just need POD to support DPDK.
09:03:04 <MatthewLi> by now for vsperf it seems not integrated with any installer, u can refer to https://build.opnfv.org/ci/view/vswitchperf/
09:03:17 <MatthewLi> runs on intel POD3
09:03:58 <Yaoguang_Wang> zhihui_wu: Yes. DPDK support in guest is easy. So far, we need to find the support in POD. Kubi will find the way :)
09:04:54 <Yaoguang_Wang> MatthewLi:  Great
09:05:23 <zhihui_wu> Thanks to all.
09:05:38 <kubi> zhihui_wu: Yaoguang_Wang MatthewLi: thank you for your great suggestion.  i will check with installer team and Vsperf team. it is a real good question
09:06:24 <kubi> #topic scenario validation
09:06:33 <kubi> #info yardstick will provide some test cases for scenario validation.
09:06:45 <kubi> here is functest defined test cases which are sliced by tier.
09:06:52 <kubi> #link https://git.opnfv.org/cgit/functest/tree/ci/testcases.yaml
09:07:10 <kubi> here is Test Release Criteria #link https://wiki.opnfv.org/display/SWREL/Test+Release+Criteria
09:07:33 <kubi> so, we need to define the tiers like functest
09:07:50 <kubi> #link https://etherpad.opnfv.org/p/yardstick_scenario_validation
09:07:55 <kubi> here is a draft
09:08:08 <kubi> please feel free to give comments
09:08:42 <kubi> #action all review the scenario validation draft and give comments
09:08:43 <MatthewLi> I am interested and would like to contribute :)
09:08:53 <kubi> MatthewLi: that's great
09:09:27 <kubi> #topic others
09:09:37 <kubi> anything else?
09:10:40 <kubi> thank you for today. keep in touch:)
09:10:50 <Yaoguang_Wang> Thanks kubi
09:11:02 <kubi> keep active code review, thank you, everyone
09:11:05 <MatthewLi> thanks
09:11:08 <kubi> #endmeeting