08:31:41 <rbbratta> #startmeeting yardstick work meeting
08:31:41 <collabot> Meeting started Mon Jul 24 08:31:41 2017 UTC.  The chair is rbbratta. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:31:41 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:31:41 <collabot> The meeting name has been set to 'yardstick_work_meeting'
08:31:49 <rbbratta> #topic roll call
08:31:51 <rbbratta> #info Ross
08:31:57 <Mingjiang> #info Rex
08:31:58 <abhijitsinha> #info Abhijit
08:32:08 <JingLu> #info JingLu
08:32:10 <JackChan> #info Jack
08:32:11 <Kanglin> #info Kanglin
08:32:32 <rbbratta> #topic action item follow up
08:32:41 <kubi001> #info kubi
08:33:00 <rbbratta> #action Ross figure out how to tag Jira tasks for testcases
08:33:15 <rbbratta> still need to figure out Jira task for testcases, unless anyone else has ideas
08:33:43 <rbbratta> #info JingLu file ticket to delete danube.3.0 docker container
08:33:50 <rbbratta> are we ready to delete Danube 3.0 docker?
08:34:12 <JingLu> Dovetail are using danube.3.1 now, I will send mail to helpdesk today
08:34:31 <rbbratta> #action JingLu delete danube.3.0 docker
08:34:48 <rbbratta> #info Yardstick team met with Bottlenecks team to discuss scale-up/scale-out
08:35:05 <rbbratta> #action Ross create scale-up/scale-out testcase template for bottlenecks team
08:35:46 <rbbratta> any other feeadback on scale-up/scale-out meeting?
08:36:46 <rbbratta> okay another other action item feedback?
08:37:18 <rbbratta> #topic Euphrates MS5 feature freeze
08:37:26 <kubi001> will we still have  joint meeting with bottlenecks this week?
08:38:15 <rbbratta> I don't have anything now
08:38:48 <rbbratta> maybe for next meeting
08:39:26 <kubi001> rbbratta: do you mean "next week"?
08:39:56 <rbbratta> the next Wednesday Bottlenecks meeting
08:40:18 <kubi001> ok, got it
08:40:39 <rbbratta> ah, bottlenecks team joined
08:41:09 <rbbratta> I don't have testcase template created yet.
08:42:14 <rbbratta> I will try to create the testcase and email it out, then maybe we can discuss next bottlenecks meeting on wednesday?
08:42:37 <gabriel_yuyang> rbbratta: ok, let's discuss the in next week bottlenecks meeting about the detailed calling process
08:43:27 <gabriel_yuyang> I will also work on some proposal based on your template
08:43:32 <rbbratta> gabriel_yuyang: okay
08:44:07 <Mingjiang> does this scale test case need features to be added in the framework?
08:44:32 <rbbratta> for scale-up we just use heat flavor creation
08:45:50 <Mingjiang> that's great. it would not affect MS5.
08:46:32 <rbbratta> it needs all the other NSB features
08:46:47 <rbbratta> but the actual scale-up is just the flavor
08:48:20 <OPNFV-Gerrit-Bot> Ross Brattain proposed yardstick: add VNF scale-up template  https://gerrit.opnfv.org/gerrit/37991
08:48:59 <rbbratta> okay, well I just published the template
08:49:15 <rbbratta> #info https://gerrit.opnfv.org/gerrit/37991  scale-up
08:50:51 <rbbratta> we would need to iterator over both vcpus and mem.  there is a minimum number of vcpus and RAM for the VNF to use.
08:51:51 <rbbratta> we also have to adjust the VNF config to add more load-balancers to scale out over more cores, but that is VNF code.
08:55:07 <rbbratta> okay, any other MS5 topics?
08:55:16 <rbbratta> I see GUI patches were published
08:55:21 <JackChan> yes
08:55:52 <rbbratta> I submitted a bunch of ansible playbooks to start to replace the shell scripts with ansible.
08:55:57 <JackChan> I have just upload the gui patch and its deployment.
08:56:22 <JackChan> For migration test case, I am still working on it.
08:56:52 <JackChan> rbbratta: That's cool!
08:56:57 <rbbratta> I have to submit PROX NVFi characterization code and NSB changes
08:57:28 <rbbratta> the ansible code was tested briefly, it is designed to replace load_images.sh and clean_images.sh
08:57:36 <rbbratta> and to use Dockerfile templates
08:58:07 <rbbratta> There was a collectd question for KPIs
08:58:43 <rbbratta> It looks like not many installers will have collectd installed by default, so we may not be able to get NVFi stats
08:59:27 <rbbratta> There is a script to install collectd from git source inside the VNF, but I don't think we have code to install onto the NVFi compute nodes.
08:59:41 <kubi001> can we install the collected by ansible script?
09:00:14 <rbbratta> I planed to convert the shell script to ansible, but haven't done that yet.  We could use ansible playbook to install in VNF.
09:00:51 <rbbratta> I don't think I have ansible code to connect to the hypervisors.  We need to figure out how to get hypervisor ip/user from OpenStack
09:01:13 <rbbratta> unless the installer code, like fuel code can find compute nodes credentials
09:01:30 <kubi001> so we can get the info from VMs but hosts
09:01:58 <rbbratta> yes that it is the current state
09:03:45 <rbbratta> if the hosts have collectd pre-installed, then there may be code to connect from yardstick docker to AMPQ collectd socket, but IP address may have to be manually set.
09:04:39 <Mingjiang> do we just need the pod node ip to do this?
09:05:43 <kubi001> well, that's a problem
09:05:47 <rbbratta> to connect to AMPQ you need IP.  I think the collectd.sh shell script creates an admin user for rabbitmqctl
09:06:15 <rbbratta> https://gerrit.opnfv.org/gerrit/#/c/35343/15/yardstick/network_services/nfvi/collectd.sh
09:06:22 <rbbratta> #link https://gerrit.opnfv.org/gerrit/#/c/35343/15/yardstick/network_services/nfvi/collectd.sh
09:07:27 <rbbratta> okay, so collectd.sh was already merged
09:08:50 <rbbratta> collectd.sh also installed Intel RDT CAT/CMT tool to monitor LLC utilization https://github.com/01org/intel-cmt-cat.git
09:09:05 <rbbratta> so that is fun
09:09:34 <rbbratta> #topic AoB
09:09:59 <Mingjiang> ha test cases in Euphrates
09:10:25 <Mingjiang> kanglin: ping
09:10:39 <Kanglin> Fu Qiao called me the attend HA meeting this Wednesday to disscuss ha test cases in release E
09:11:31 <Kanglin> And I've prepared drafts of three ha test cases
09:11:51 <Kanglin> https://etherpad.opnfv.org/p/yardstick_ha_tc013
09:11:53 <Kanglin> https://etherpad.opnfv.org/p/yardstick_ha_tc014
09:11:56 <Kanglin> https://etherpad.opnfv.org/p/yardstick_ha_tc015
09:12:28 <JackChan> rbbratta: what about the yardstick.conf bug in danube.3.1?
09:13:08 <Kanglin> I'll attend HA meeting this Wednesday to confirm these cases.
09:13:12 <rbbratta> JackChan: do we know the extent and proposed workaround?
09:13:35 <rbbratta> Kanglin: do these testcases need framework changes?
09:14:23 <Mingjiang> Kanglin: that's great. i'd like to attend the meeting, too.
09:14:44 <JackChan> rbbratta: yes, we can set a volume mapping or set yardstick.conf manually.
09:15:31 <Kanglin> rbbratta: the intermediate variable patch need to be merged.
09:15:40 <Mingjiang> JackChan: you may create a wiki page about how to workaroud it
09:15:54 <Kanglin> Qiujuan have submitted a new patch based on your comment last time.
09:16:01 <rbbratta> JackChan: and what is the current state, how severe if we don't workaround?
09:16:01 <Kanglin> https://gerrit.opnfv.org/gerrit/#/c/35307/
09:16:41 <JackChan> rbbratta: Did you sent the Email to David about tag issue?
09:16:53 <rbbratta> #info https://gerrit.opnfv.org/gerrit/#/c/35307/ change required HA framework, everyone please review
09:17:04 <JackChan> Mingjiang: yes, will do it.
09:17:32 <rbbratta> JackChan: I didn't.  we need a write up about the severity, so we have justification.
09:17:54 <JackChan> rbbratta: if we do not set yardstick.conf, the task will shutdown.
09:18:24 <rbbratta> JackChan: okay so we don't function without yardstick.conf.  How is the error message, will the use know to create yardstick.conf after task shutdown?
09:19:30 <JackChan> rbbratta: sadly no, there is no any error message due to the atexit.
09:20:50 <JackChan> rbbratta: atexit will catch the exception, and then execute error handler.
09:21:12 <rbbratta> JackChan: okay so it is severe, please write up this description of the sequence of events, also include proposed workarounds and I will send email to David
09:22:22 <JackChan> okey, I will write it asap.
09:22:46 <rbbratta> once we email David, there will probably be questions about root cause of how the defect was introduced.
09:24:02 <JackChan> rbbratta: which patch introduced?
09:24:23 <rbbratta> JackChan: which patch introduced the issue?
09:25:11 <JackChan> https://gerrit.opnfv.org/gerrit/#/c/37483/
09:25:54 <rbbratta> then the questions is, why was the patched merged without sufficient testing?
09:26:28 <rbbratta> all Danube 3.0 tests were supposed to be completed on July 12th, the patch was merged July 14th.
09:27:06 <JackChan> rbbratta: it's my fault. I have this file locally, so I did't notice this bug.
09:28:02 <rbbratta> the issue is more about the process, why was the patched merged so late in the release process.
09:28:48 <rbbratta> The patch was reviewed by JingLu and Mingjiang and merged by Mingjiang
09:29:24 <rbbratta> and then I tagged the release
09:30:27 <Mingjiang> yes, i think this patch is kind of delayed and not tracked as a high priority.
09:31:05 <JingLu> And the yardstick.conf is also exist in the CI, so CI jobs didn't reveal this bug either.
09:31:27 <Mingjiang> it's not right to merge it after July 12th
09:32:57 <rbbratta> I agree, so we need to take steps for next release to be more careful
09:33:49 <rbbratta> okay, I think we have a good idea about what happened, so we can start the discussion with David.
09:34:50 <Mingjiang> also we don't have a smoke test locally which may discover issue like this early
09:36:07 <rbbratta> yes, we need more testing.
09:36:49 <rbbratta> #action Ross create plan for more CI testing for Euphrates
09:36:50 <JackChan> Mingjiang: yes, need more basic test from starting a docker container.
09:37:13 <rbbratta> we can discuss more ideas at future meetings
09:37:29 <rbbratta> I think we should wrap up this meeting
09:37:47 <Mingjiang> agree
09:38:23 <rbbratta> okay, anything else?
09:38:57 <rbbratta> #endmeeting