08:31:34 <rbbratta> #startmeeting yardstick work meeting
08:31:34 <collabot> Meeting started Mon Jul 17 08:31:34 2017 UTC.  The chair is rbbratta. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:31:34 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:31:34 <collabot> The meeting name has been set to 'yardstick_work_meeting'
08:31:45 <rbbratta> #topic roll call
08:31:57 <rbbratta> #info Ross
08:31:59 <JackChan> #info Jack
08:32:15 <JingLu> #info Jing_Lu
08:32:16 <Kanglin> #info Kanglin
08:32:58 <deepaks> #info deepak
08:33:14 <Mingjiang5> #info Rex
08:33:16 <Ace__> #info Ace
08:34:14 <rbbratta> #topic action item follow up
08:34:49 <rbbratta> I think the only action item was anteater, which is now enabled.
08:35:11 <rbbratta> Any other agenda items before we proceed?
08:35:58 <rbbratta> #topic Danube 3.1
08:36:16 <rbbratta> #info Danube 3.1 was git tagged
08:36:27 <rbbratta> did we build docker for danube.3.1?
08:36:42 <rbbratta> We had a request to delete the docker.3.0 image to prevent confusion
08:36:45 <JingLu> I've tag our docker image with tag danube.3.1 this morning
08:37:18 <rbbratta> can we delete danube.3.0 docker image?
08:37:49 <JingLu> I didn't have the right to delete
08:38:40 <rbbratta> deleting it won't affect Dovetail?
08:39:15 <rbbratta> we would have to file a ticket with helpdesk to ask them to delete it
08:39:25 <JingLu> We have informed Dovetail, they will use 3.1
08:39:48 <rbbratta> okay, so we just need action item to request danube.3.0 container deletion
08:40:09 <rbbratta> JingLu: can you file ticket with helpdesk?
08:40:41 <JackChan> rbbratta: so we do not need danube.3.0 docker image anymore?
08:40:45 <rbbratta> Have people been following the docker version thread on test-wg
08:40:51 <JingLu> rbbratta okay, I will do it after meeting
08:41:16 <rbbratta> JackChan: Alec Hothan asked if we could remove danube.3.0 to avoid confusion
08:41:39 <rbbratta> since the docker danube.3.0 container does not exactly match the danube.3.0 git tag, it is misleading
08:41:48 <JackChan> rbbratta: ok
08:41:51 <rbbratta> docker containers should always match the git tag
08:42:10 <rbbratta> but they we going to discuss all this versioning business at some test-wg meeting soon.
08:42:55 <rbbratta> we will probably come up with a way to include git commit id in every container, so we can match container to exact git commit
08:43:46 <rbbratta> #action JingLu file ticket to delete danube.3.0 docker container
08:44:02 <rbbratta> #info https://wiki.opnfv.org/display/meetings/Test+Working+Group+Weekly+Meeting  July 27 agenda: Docker container management (versioning and build workflow)
08:44:44 <rbbratta> #info please check the test-wg July 27th agenda for details on docker container versioning proposals
08:44:52 <rbbratta> next topic, the release itself.
08:45:39 <rbbratta> I noticed  https://gerrit.opnfv.org/gerrit/#/c/37483/  was merged on the last day
08:46:09 <rbbratta> was there a reason we merged this?
08:47:00 <OPNFV-Gerrit-Bot> Deepak S proposed yardstick: Fix adding right deb repo based on the distro we are running on  https://gerrit.opnfv.org/gerrit/35353
08:47:03 <JackChan> rbbratta: we want to support centos in danube.3.0
08:47:43 <JackChan> since centos/redhat is very important to us.
08:48:00 <rbbratta> was it tested?
08:48:23 <JackChan> yes, I tested it in redhat.
08:48:43 <rbbratta> as I understand the release procedures, testing was supposed to have been completed on July 12th.  We merged after that date.
08:49:10 <rbbratta> merged at the last minute is very risky
08:49:17 <rbbratta> *merging
08:49:46 <JackChan> rbbratta: ok, maybe next time, we should check it earlier.
08:50:44 <rbbratta> I would expect critical issues to be added to meeting agendas and discussed before release.  Because in general we should be reducing risk as we prepare for release
08:51:27 <rbbratta> okay, we can work on process more for upcoming releases
08:52:08 <rbbratta> but we have had various issues with yardstick CI breaking all through danube releases.  mostly due to git branch issues.
08:53:17 <rbbratta> for Euphrates I suspect there will be increased checking by release managers to make sure we don't break CI.
08:54:31 <rbbratta> any other issues/concerns with Danube release?  We are finally done!  :)   no more Danube, we can focus on Euphrates.
08:55:03 <rbbratta> next topic
08:55:15 <rbbratta> #topic unittests
08:55:33 <rbbratta> there was some unittest breakage due to the heat change.
08:56:10 <rbbratta> we have been observing some differences between locally-run unittest results and Jenkins results.
08:56:25 <rbbratta> different tests fail in a local environment.
08:56:48 <rbbratta> I'm trying to narrow it down.  Has anyone else seen unittest issues?
08:57:08 <JackChan> rbbratta: yes, and some unit test case failed randomly.
08:58:05 <JackChan> our local result is different from ci sometimes.
08:58:37 <rbbratta> I'm running on Fedora, I assume the Jenkins systems are Ubuntu 14.04, but I can check.
08:58:48 <JingLu> Seems failures are most on NSB unittest
08:58:58 <rbbratta> I also though about adding more Jenkins slaves, maybe I can add a system to pharos to run more unittests.
08:59:38 <kubi001> #info kubi
08:59:42 <kubi001> sorry for the late
09:00:09 <rbbratta> The other issues is unittest speed.  I'm trying to speed up the unittests.  We need to modify run_tests.sh so we don't duplicate coverage on both py27 and py3.
09:00:54 <rbbratta> We also need to watch out for uncessary delays.  I have gone through all the results locally and found the slowest tests.
09:01:27 <rbbratta> All the slowest tests are using hardcoded time.sleep(5)  calls.  These sleeps should not be run in unittests, we need to mock these out.
09:02:31 <Mingjiang5> agree
09:02:57 <JackChan> rbbratta: but if we do not sleep, some case will be failed.
09:03:00 <rbbratta> I will try to add documentation somewhere, or make sure when we code-review unittests that we check for time.sleep() and make sure they are mocked
09:03:32 <rbbratta> JackChan: then we need to fix those tests.  The calls to external resources should be mocked.
09:04:11 <JackChan> rbbratta: got it.
09:04:23 <rbbratta> There are examples in the fixes, for example I added a while loop to wait until a queue is full before continuing
09:05:38 <rbbratta> On local tests I reduced the time from 1m30s to 3s
09:06:12 <rbbratta> unittests should be nearly instantaneous.
09:06:31 <rbbratta> so please review the remaning unittest fixes so we can merge.
09:06:43 <JackChan> ok
09:07:26 <rbbratta> next topic
09:07:46 <rbbratta> #topic Euphrates MS5 feature freeze
09:08:42 <rbbratta> I have been updating Jira, but we all need to focus on MS5.  We will lower merge criteria somewhat so we can merge all the features ever if not completely working
09:09:26 <rbbratta> Please update at meeting if there are any features that are blocked or delayed or that otherwise might miss MS5.
09:09:40 <rbbratta> MS5 is July 28th
09:10:10 <rbbratta> #info Please update at meeting if there are any features that are blocked or delayed or that otherwise might miss MS5 on July 28th
09:10:23 <JackChan> #link https://gerrit.opnfv.org/gerrit/#/c/33431/
09:10:27 <Mingjiang5> i think i have a task about fetch SUT infomation
09:11:06 <JackChan> This is live migration, I need to write more unittest case. But I have upload the gui.
09:11:44 <rbbratta> #action Ross review https://gerrit.opnfv.org/gerrit/#/c/33431/
09:12:24 <JackChan> I will do my best to write unittest case. but if I don't have enough time. we will need to merge it first.
09:12:48 <rbbratta> yes, we might have to waive unittest or coverage requirements
09:13:45 <JackChan> rbbratta: yes
09:14:13 <rbbratta> is yardstick CI expected to be stable during MS5?  Can we break jenkins for a bit?
09:15:17 <rbbratta> okay, but in general we expect everyone to make a best effort to get unittests and coverage passing before MS5
09:15:43 <JingLu> I think there is no requirement about CI during MS5. What do you want to do with it?
09:15:58 <JackChan> rbbratta: we will try our best effort to write unit test case.
09:16:26 <rbbratta> JingLu: when we merge everything for MS5 I expect yardstick tests to break until we fix everything
09:16:53 <rbbratta> so the installer tests might break
09:17:14 <JingLu> rbbratta got it.
09:17:59 <rbbratta> but that might be expected, how has feature freeze worked in previous releases?
09:18:44 <rbbratta> okay, maybe we can discuss more in next meeting.
09:19:05 <rbbratta> the last topic was scale-up/scale-out, but we can push that to next meeting.
09:19:33 <rbbratta> based on requirements I expect we can implement scale-up/scale-out using bottlenecks, but we have to ask bottlenecks people.
09:19:47 <rbbratta> #topic AoB
09:20:00 <kubi001> I noticed that jing have a action items about  deleting danube.3.0 docker container, I agree with that. But  that would make CVP.0.2.0 failed.  From my perspective, we can delete the danube.3.0 image one week later, then, they will have a CVP.0.3.0 which will integrate Yardstick Danube.3.1, so they don't need the Yardstick Danube.3.0 any more
09:20:27 <rbbratta> kubi001: okay
09:20:36 <kubi001> thanks
09:21:08 <JingLu> I will file the ticket one week later
09:21:13 <rbbratta> #info delay docker danube.3.0 container deletion until after CVP.0.3.0
09:22:00 <rbbratta> anything else?
09:22:29 <rbbratta> going twice
09:22:45 <rbbratta> #endmeeting