08:31:41 <rbbratta> #startmeeting yardstick work meeting 08:31:41 <collabot> Meeting started Mon Jul 24 08:31:41 2017 UTC. The chair is rbbratta. Information about MeetBot at http://wiki.debian.org/MeetBot. 08:31:41 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic. 08:31:41 <collabot> The meeting name has been set to 'yardstick_work_meeting' 08:31:49 <rbbratta> #topic roll call 08:31:51 <rbbratta> #info Ross 08:31:57 <Mingjiang> #info Rex 08:31:58 <abhijitsinha> #info Abhijit 08:32:08 <JingLu> #info JingLu 08:32:10 <JackChan> #info Jack 08:32:11 <Kanglin> #info Kanglin 08:32:32 <rbbratta> #topic action item follow up 08:32:41 <kubi001> #info kubi 08:33:00 <rbbratta> #action Ross figure out how to tag Jira tasks for testcases 08:33:15 <rbbratta> still need to figure out Jira task for testcases, unless anyone else has ideas 08:33:43 <rbbratta> #info JingLu file ticket to delete danube.3.0 docker container 08:33:50 <rbbratta> are we ready to delete Danube 3.0 docker? 08:34:12 <JingLu> Dovetail are using danube.3.1 now, I will send mail to helpdesk today 08:34:31 <rbbratta> #action JingLu delete danube.3.0 docker 08:34:48 <rbbratta> #info Yardstick team met with Bottlenecks team to discuss scale-up/scale-out 08:35:05 <rbbratta> #action Ross create scale-up/scale-out testcase template for bottlenecks team 08:35:46 <rbbratta> any other feeadback on scale-up/scale-out meeting? 08:36:46 <rbbratta> okay another other action item feedback? 08:37:18 <rbbratta> #topic Euphrates MS5 feature freeze 08:37:26 <kubi001> will we still have joint meeting with bottlenecks this week? 08:38:15 <rbbratta> I don't have anything now 08:38:48 <rbbratta> maybe for next meeting 08:39:26 <kubi001> rbbratta: do you mean "next week"? 08:39:56 <rbbratta> the next Wednesday Bottlenecks meeting 08:40:18 <kubi001> ok, got it 08:40:39 <rbbratta> ah, bottlenecks team joined 08:41:09 <rbbratta> I don't have testcase template created yet. 08:42:14 <rbbratta> I will try to create the testcase and email it out, then maybe we can discuss next bottlenecks meeting on wednesday? 08:42:37 <gabriel_yuyang> rbbratta: ok, let's discuss the in next week bottlenecks meeting about the detailed calling process 08:43:27 <gabriel_yuyang> I will also work on some proposal based on your template 08:43:32 <rbbratta> gabriel_yuyang: okay 08:44:07 <Mingjiang> does this scale test case need features to be added in the framework? 08:44:32 <rbbratta> for scale-up we just use heat flavor creation 08:45:50 <Mingjiang> that's great. it would not affect MS5. 08:46:32 <rbbratta> it needs all the other NSB features 08:46:47 <rbbratta> but the actual scale-up is just the flavor 08:48:20 <OPNFV-Gerrit-Bot> Ross Brattain proposed yardstick: add VNF scale-up template https://gerrit.opnfv.org/gerrit/37991 08:48:59 <rbbratta> okay, well I just published the template 08:49:15 <rbbratta> #info https://gerrit.opnfv.org/gerrit/37991 scale-up 08:50:51 <rbbratta> we would need to iterator over both vcpus and mem. there is a minimum number of vcpus and RAM for the VNF to use. 08:51:51 <rbbratta> we also have to adjust the VNF config to add more load-balancers to scale out over more cores, but that is VNF code. 08:55:07 <rbbratta> okay, any other MS5 topics? 08:55:16 <rbbratta> I see GUI patches were published 08:55:21 <JackChan> yes 08:55:52 <rbbratta> I submitted a bunch of ansible playbooks to start to replace the shell scripts with ansible. 08:55:57 <JackChan> I have just upload the gui patch and its deployment. 08:56:22 <JackChan> For migration test case, I am still working on it. 08:56:52 <JackChan> rbbratta: That's cool! 08:56:57 <rbbratta> I have to submit PROX NVFi characterization code and NSB changes 08:57:28 <rbbratta> the ansible code was tested briefly, it is designed to replace load_images.sh and clean_images.sh 08:57:36 <rbbratta> and to use Dockerfile templates 08:58:07 <rbbratta> There was a collectd question for KPIs 08:58:43 <rbbratta> It looks like not many installers will have collectd installed by default, so we may not be able to get NVFi stats 08:59:27 <rbbratta> There is a script to install collectd from git source inside the VNF, but I don't think we have code to install onto the NVFi compute nodes. 08:59:41 <kubi001> can we install the collected by ansible script? 09:00:14 <rbbratta> I planed to convert the shell script to ansible, but haven't done that yet. We could use ansible playbook to install in VNF. 09:00:51 <rbbratta> I don't think I have ansible code to connect to the hypervisors. We need to figure out how to get hypervisor ip/user from OpenStack 09:01:13 <rbbratta> unless the installer code, like fuel code can find compute nodes credentials 09:01:30 <kubi001> so we can get the info from VMs but hosts 09:01:58 <rbbratta> yes that it is the current state 09:03:45 <rbbratta> if the hosts have collectd pre-installed, then there may be code to connect from yardstick docker to AMPQ collectd socket, but IP address may have to be manually set. 09:04:39 <Mingjiang> do we just need the pod node ip to do this? 09:05:43 <kubi001> well, that's a problem 09:05:47 <rbbratta> to connect to AMPQ you need IP. I think the collectd.sh shell script creates an admin user for rabbitmqctl 09:06:15 <rbbratta> https://gerrit.opnfv.org/gerrit/#/c/35343/15/yardstick/network_services/nfvi/collectd.sh 09:06:22 <rbbratta> #link https://gerrit.opnfv.org/gerrit/#/c/35343/15/yardstick/network_services/nfvi/collectd.sh 09:07:27 <rbbratta> okay, so collectd.sh was already merged 09:08:50 <rbbratta> collectd.sh also installed Intel RDT CAT/CMT tool to monitor LLC utilization https://github.com/01org/intel-cmt-cat.git 09:09:05 <rbbratta> so that is fun 09:09:34 <rbbratta> #topic AoB 09:09:59 <Mingjiang> ha test cases in Euphrates 09:10:25 <Mingjiang> kanglin: ping 09:10:39 <Kanglin> Fu Qiao called me the attend HA meeting this Wednesday to disscuss ha test cases in release E 09:11:31 <Kanglin> And I've prepared drafts of three ha test cases 09:11:51 <Kanglin> https://etherpad.opnfv.org/p/yardstick_ha_tc013 09:11:53 <Kanglin> https://etherpad.opnfv.org/p/yardstick_ha_tc014 09:11:56 <Kanglin> https://etherpad.opnfv.org/p/yardstick_ha_tc015 09:12:28 <JackChan> rbbratta: what about the yardstick.conf bug in danube.3.1? 09:13:08 <Kanglin> I'll attend HA meeting this Wednesday to confirm these cases. 09:13:12 <rbbratta> JackChan: do we know the extent and proposed workaround? 09:13:35 <rbbratta> Kanglin: do these testcases need framework changes? 09:14:23 <Mingjiang> Kanglin: that's great. i'd like to attend the meeting, too. 09:14:44 <JackChan> rbbratta: yes, we can set a volume mapping or set yardstick.conf manually. 09:15:31 <Kanglin> rbbratta: the intermediate variable patch need to be merged. 09:15:40 <Mingjiang> JackChan: you may create a wiki page about how to workaroud it 09:15:54 <Kanglin> Qiujuan have submitted a new patch based on your comment last time. 09:16:01 <rbbratta> JackChan: and what is the current state, how severe if we don't workaround? 09:16:01 <Kanglin> https://gerrit.opnfv.org/gerrit/#/c/35307/ 09:16:41 <JackChan> rbbratta: Did you sent the Email to David about tag issue? 09:16:53 <rbbratta> #info https://gerrit.opnfv.org/gerrit/#/c/35307/ change required HA framework, everyone please review 09:17:04 <JackChan> Mingjiang: yes, will do it. 09:17:32 <rbbratta> JackChan: I didn't. we need a write up about the severity, so we have justification. 09:17:54 <JackChan> rbbratta: if we do not set yardstick.conf, the task will shutdown. 09:18:24 <rbbratta> JackChan: okay so we don't function without yardstick.conf. How is the error message, will the use know to create yardstick.conf after task shutdown? 09:19:30 <JackChan> rbbratta: sadly no, there is no any error message due to the atexit. 09:20:50 <JackChan> rbbratta: atexit will catch the exception, and then execute error handler. 09:21:12 <rbbratta> JackChan: okay so it is severe, please write up this description of the sequence of events, also include proposed workarounds and I will send email to David 09:22:22 <JackChan> okey, I will write it asap. 09:22:46 <rbbratta> once we email David, there will probably be questions about root cause of how the defect was introduced. 09:24:02 <JackChan> rbbratta: which patch introduced? 09:24:23 <rbbratta> JackChan: which patch introduced the issue? 09:25:11 <JackChan> https://gerrit.opnfv.org/gerrit/#/c/37483/ 09:25:54 <rbbratta> then the questions is, why was the patched merged without sufficient testing? 09:26:28 <rbbratta> all Danube 3.0 tests were supposed to be completed on July 12th, the patch was merged July 14th. 09:27:06 <JackChan> rbbratta: it's my fault. I have this file locally, so I did't notice this bug. 09:28:02 <rbbratta> the issue is more about the process, why was the patched merged so late in the release process. 09:28:48 <rbbratta> The patch was reviewed by JingLu and Mingjiang and merged by Mingjiang 09:29:24 <rbbratta> and then I tagged the release 09:30:27 <Mingjiang> yes, i think this patch is kind of delayed and not tracked as a high priority. 09:31:05 <JingLu> And the yardstick.conf is also exist in the CI, so CI jobs didn't reveal this bug either. 09:31:27 <Mingjiang> it's not right to merge it after July 12th 09:32:57 <rbbratta> I agree, so we need to take steps for next release to be more careful 09:33:49 <rbbratta> okay, I think we have a good idea about what happened, so we can start the discussion with David. 09:34:50 <Mingjiang> also we don't have a smoke test locally which may discover issue like this early 09:36:07 <rbbratta> yes, we need more testing. 09:36:49 <rbbratta> #action Ross create plan for more CI testing for Euphrates 09:36:50 <JackChan> Mingjiang: yes, need more basic test from starting a docker container. 09:37:13 <rbbratta> we can discuss more ideas at future meetings 09:37:29 <rbbratta> I think we should wrap up this meeting 09:37:47 <Mingjiang> agree 09:38:23 <rbbratta> okay, anything else? 09:38:57 <rbbratta> #endmeeting