08:00:09 <morgan_orange> #startmeeting Functest weekly meeting 3rd of October
08:00:09 <collabot> Meeting started Tue Oct  3 08:00:09 2017 UTC.  The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:09 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:09 <collabot> The meeting name has been set to 'functest_weekly_meeting_3rd_of_october'
08:00:14 <morgan_orange> #info topic role
08:00:20 <morgan_orange> #info Morgan Richomme (proxy Jose)
08:00:22 <CristinaPauna> #info Cristina Pauna
08:00:30 <juhak> #info Juha Kosonen
08:00:33 <viktor_t> #info Viktor Tikkanen
08:00:39 <morgan_orange> #info agenda: https://wiki.opnfv.org/display/functest/Functest+5.+Meeting
08:01:00 <depo> #info Delia Popescu
08:01:09 <morgan_orange> #info Cédric
08:01:29 <morgan_orange> #topic action point follow-up
08:01:35 <morgan_orange> #info AP1: jose_lausuch send emails to applicants
08:01:40 <morgan_orange> #info done
08:01:46 <morgan_orange> #info AP2: jose_lausuch add euphrates support in functest-alpine.sh
08:01:52 <morgan_orange> #info done
08:01:58 <morgan_orange> #info AP3: remove ericsson-virtual-pod1bl01 from functest alpine pods
08:02:04 <morgan_orange> #info done
08:02:10 <morgan_orange> #info AP4: jose_lausuch build it again and push it
08:02:15 <morgan_orange> #info done, build done though hook on docker hub (not using releng, clearification shoudl be done on the way to build docker after E with releng)
08:02:24 <morgan_orange> any question, comment on thath?
08:02:40 <morgan_orange> #topic Release status
08:02:50 <morgan_orange> #info theory release date: 6th of October
08:03:18 <morgan_orange> #info release meeting this afternoon, I will proxy Jose (Currently travelling for Suse (non OPNFV releated activities) )
08:03:43 <morgan_orange> #info vyos => move to 5.1, patch submitted and merged this morning
08:03:54 <morgan_orange> #link https://gerrit.opnfv.org/gerrit/#/c/43995/
08:04:11 <morgan_orange> #info next step troubleshoot on a community lab
08:04:28 <morgan_orange> #action morgan_orange grant access to Orange Community Lab 3 (problem is reproducible) to Shuya_ool
08:04:47 <morgan_orange> #info once OK on community lab => enable on Master  => should be ok for 5.1
08:04:55 <morgan_orange> are you ok with that Shuya_ool?
08:05:07 <morgan_orange> I will send a mail to connect you to the admin of the community lab
08:05:44 <ollivier> regarding jjobs, we must fix the clean... the grep filter catches every containers.
08:06:48 <morgan_orange> #info issue on container management - wild clean catching all the containers
08:07:32 <morgan_orange> #link https://git.opnfv.org/releng/tree/jjb/functest/functest-cleanup.sh
08:07:58 <morgan_orange> ollivier: I action Jose on that?
08:08:32 <ollivier> I would say yes.
08:08:40 <ollivier> trivial patch
08:09:12 <morgan_orange> #action jose_lausuch fix cleaning docker mechanism
08:09:51 <morgan_orange> I know that he has not lots of time due to new Suse activities, but I am pretty sure he will do it quickly
08:10:24 <morgan_orange> #info vping_userdata: we used to have an old bug in vping_userdata ...all the results were PASS whatever the result of the patch
08:10:48 <morgan_orange> #info several patchs done to fix the issue...
08:11:15 <morgan_orange> #info we should have got a FAIL status on the nightly run, but apparently not..need to check
08:11:49 <morgan_orange> #info Cedric's morning patch (adding a router in vping_userdata and not only in vping_ssh) should fix
08:12:01 <morgan_orange> #info but we have weeks of tests with False positive..
08:12:48 <morgan_orange> #action morgan_orange check nighlty run / vping_userdata and check why Daisy run is PASS (https://build.opnfv.org/ci/view/functest/job/functest-daisy-baremetal-daily-euphrates/) run supposed to integrated the reporting fix but not the router fix
08:12:57 <ollivier> I suppose that no OPNFV guideline defines if metadata must work in a L2 network.
08:13:31 <morgan_orange> I think the guidelines are limited to the OpenStack version...and it is not always followed.. :)
08:13:35 <ollivier> I asked Steven if snaps allows creating a subnet without any gateway
08:13:53 <morgan_orange> #info remaining erros on Tempest/Rally/Snaps
08:14:05 <morgan_orange> #info all the tests are oK on Daisy (Koola Open Stack)
08:14:18 <morgan_orange> #info including the live migration test in Rally (Nova)
08:14:30 <morgan_orange> #info this test is FAIL in lots of scenarios/installers
08:14:35 <ollivier> We could check if Daisy supports that and if DHCP relays the metadata even if the router is defined but not created.
08:15:14 <ollivier> But now we rely on l3-agent for this part.
08:15:33 <morgan_orange> #action morgan_orange contact Daisy team to get details on metadata management
08:15:50 <morgan_orange> juhak: did you have time to go through the different scenarios
08:16:22 <juhak> rally_sanity on apex: at least correction for this was not in build: https://jira.opnfv.org/browse/APEX-525
08:16:53 <juhak> but live migration fails in a similar way also in case other installers
08:17:53 <juhak> I don't have euphrates env where I could investivate live migr problems
08:18:25 <juhak> on compass there is the segmentation failure case, https://jira.opnfv.org/browse/FUNCTEST-876
08:18:31 <ollivier> It fails in Compass Danube 2.0 as well.
08:18:38 <morgan_orange> #info pending Apex bugs ( https://jira.opnfv.org/browse/APEX-525) preventing nova migration => leading to rally_sanity error status
08:18:59 <juhak> on apex danube 2.0 its is ok
08:19:11 <ollivier> ok.
08:19:21 <ollivier> Then the test seems relevant
08:20:03 <morgan_orange> #info livre migration on apex/Danube 2.0 was ok
08:20:08 <morgan_orange> #info segmentation failure on Compass (https://jira.opnfv.org/browse/FUNCTEST-876)
08:20:19 <juhak> on joid the failure is clear: "rack-12-m1 is not on shared storage: Shared storage live-migration requires either shared storage or boot-from-volume with no local disks"
08:20:22 <morgan_orange> #info test on compass/Danube 2.0 fail
08:20:48 <juhak> at the moment live migration tests are always enabled when more than 1 compute available, https://gerrit.opnfv.org/gerrit/#/c/41503/
08:20:54 <morgan_orange> #info error on joid => due to infra "rack-12-m1 is not on shared storage: Shared storage live-migration requires either shared storage or boot-from-volume with no local disks"
08:21:03 <morgan_orange> #info at the moment live migration tests are always enabled when more than 1 compute available, https://gerrit.opnfv.org/gerrit/#/c/41503/
08:21:25 <morgan_orange> I think there are lots strange thing son joid (Tempest success rate ~ 75% where it is above 95% for almost all the others
08:21:48 <morgan_orange> so for me the test is relevant, was working on several installers in Danube, still working on Daisy..
08:21:57 <morgan_orange> troubleshooting needed on installer side
08:22:01 <juhak> In general, many of the failed tempest cases used to be blacklisted: https://gerrit.opnfv.org/gerrit/#/c/36907/
08:22:21 <morgan_orange> #info juhak many of the failed tempest cases used to be blacklisted: https://gerrit.opnfv.org/gerrit/#/c/36907/
08:22:49 <ollivier> Agree with Morgan. from the time being the failure seems relevant
08:23:12 <morgan_orange> but there are several scenarios where the tests are all OK...
08:23:28 <ollivier> Let see if Apex fixes it or not.
08:24:47 <morgan_orange> what's your view regarding blacklisting? I think that our goal is to be neutral..if it works on some scenario and if there is no upstream bug to explicitely exclude some cases (which is always possible due to intergation)
08:24:58 <ollivier> #agree
08:25:35 <morgan_orange> juhak: do you think there are still relevant bugs from Danube to exclude some cases in some scenarios?
08:25:41 <morgan_orange> not bugs from Danube
08:25:50 <morgan_orange> but upstrteam bugs referenced in Danube
08:25:51 <ollivier> if it MUST be PASS, the best way is to decrease the ratio asked.
08:26:38 <juhak> now there are several failed cases that used to be blacklisted
08:27:01 <juhak> the reason for these failures should now be investigated
08:27:16 <morgan_orange> juhak: yes but I assume last time we associated a reason for blacklisting, is teh reason still valid in ocata?
08:27:37 <morgan_orange> I think it shall be reference somewhere in Danube release note...
08:27:49 <ollivier> Do we have updated the blacklist for E?
08:28:34 <morgan_orange> ollivier: we will have to (action Jose), if the date is still the 6th, it shall be done before...for the moment it has not been initiated
08:28:55 <morgan_orange> wait and see for the release meeting this afternoon (I will proxy Jose)
08:29:02 <morgan_orange> and will give you feedback
08:29:36 <morgan_orange> #action juhak check if referenced Danube upstream bugs (motivating tempest black listing) are still valid or not
08:29:51 <ollivier> I'm still convinced that we should decrease ratio instead of blacklisting in this case.
08:30:09 <ollivier> But I'm fine if the testcase fails.
08:30:18 <ollivier> We should give the right status
08:30:23 <morgan_orange> yes that was the approach in previous version..we accepted a 90%
08:30:39 <morgan_orange> today for example if you look at Tempest results
08:30:56 <morgan_orange> http://testresults.opnfv.org/reporting/euphrates/functest/tempest-apex.html
08:31:03 <morgan_orange> http://testresults.opnfv.org/reporting/euphrates/functest/tempest-compass.html
08:31:09 <morgan_orange> http://testresults.opnfv.org/reporting/euphrates/functest/tempest-daisy.html
08:31:16 <morgan_orange> http://testresults.opnfv.org/reporting/euphrates/functest/tempest-fuel.html
08:31:21 <morgan_orange> http://testresults.opnfv.org/reporting/euphrates/functest/tempest-joid.html
08:31:42 <morgan_orange> you can see that most of the scenarios (scenarii :)) are above 95%
08:32:21 <morgan_orange> but Apex, Daisy, Compass and Fuel have scenarios at 100%,
08:32:27 <morgan_orange> so the criteria sounds fine for me...
08:32:51 <juhak> I agree
08:33:39 <morgan_orange> #info snaps error in scenarios. we can see that some scenarios have issue on snaps_smoke and snaps_healthcheck
08:33:42 <ollivier> but it seems false that results differ on same scenario (idem in Latin)
08:34:16 <morgan_orange> yep it is even strange...
08:34:47 <morgan_orange> and even sometime form 1 run to another see apex/odl-nofeature-ha
08:36:16 <ollivier> This is the typical case where we could blacklist. But if ODL doesn't work as well as Neutron, we should note that.
08:37:02 <morgan_orange> #info status on the feature project onboarded in Functest
08:37:23 <morgan_orange> #link https://wiki.opnfv.org/display/functest/Functest+feature+integration+status
08:37:34 <morgan_orange> #info barometer now OK (50 minutes...longest test case)
08:37:50 <morgan_orange> #info parser OK on 2 installer (but not on apex)
08:38:04 <morgan_orange> #info domino ok (but ..does not do lots of things..)
08:38:20 <morgan_orange> sfc working on xci
08:38:40 <morgan_orange> need to get visual confirmation form Manuel
08:39:02 <morgan_orange> #info still issues on doctor, bgpvpn, promise
08:39:23 <morgan_orange> as they are no more part of our criteria, it is not a big issue to keep them failing..
08:40:01 <morgan_orange> #info on vnf integration: vyos postponed to 5.1 (see previous logs)
08:40:03 <ollivier> It's a release issue anyway.
08:40:21 <morgan_orange> #info orchestra openims OK since last night (resizing of the image did the job)
08:40:47 <morgan_orange> #info orchestra_clearwaterims used to be OK but removing icmp rules lead to regression (reenable this morning) => should be OK
08:41:11 <morgan_orange> #info thanks to orchestra team for the troubleshooting
08:41:29 <ollivier> #agree
08:41:30 <morgan_orange> #info note we will probably redeclare them as functest cases as the code is hosted in functest
08:41:36 <ollivier> #agre
08:41:39 <ollivier> #agree
08:41:47 <morgan_orange> #info for F release, we should probably apply the same way than for feautres
08:41:59 <morgan_orange> #info what is hosted in Functest => functest otherwise host the code in your repo
08:42:13 <morgan_orange> #info it is clean for the feature now, vnf shall follow...
08:42:31 <morgan_orange> any question on features, vnf?
08:42:41 <ollivier> But they must follow the same rules (pylint, unit tests)
08:42:47 <morgan_orange> yes...
08:43:06 <morgan_orange> the issue with vnf is that sometimes there is no associated project
08:43:29 <ollivier> Yes. It's fine for a first step. We will see during F cycle.
08:43:53 <morgan_orange> #info release note to be written
08:44:03 <morgan_orange> #action jose_lausuch initiate the release note
08:44:18 <morgan_orange> any question regarding Euphrates?
08:44:40 <ollivier> Coud you pelase check https://gerrit.opnfv.org/gerrit/#/c/43689/ ?
08:44:44 <morgan_orange> I think we would be OK to release on the 6th, of course wa are lacking iterations, but the framework is ready
08:44:59 <morgan_orange> ollivier: I will do after the meeting
08:46:10 <morgan_orange> I do not see blocking points (especially thanks to  the express fix on vping_userdata) for Euphrates..of course we may have more time for more troubleshooting but as we have scenarios already OK..I do not see any red light
08:46:15 <morgan_orange> do you see any?
08:46:29 <ollivier> For E, it would have been great that ARM follows the rules.
08:46:54 <CristinaPauna> olliver: please elaborate
08:47:07 <ollivier> We can't maintain both containers.
08:47:34 <ollivier> It would have been great you switched to Alpine as well.
08:47:47 <CristinaPauna> olliver: I know, and we tried
08:48:08 <CristinaPauna> I don't think that makes us out of rules
08:48:22 <ollivier> Agree.
08:48:53 <morgan_orange> #topic AoB
08:49:01 <morgan_orange> #info 2 new interns joined Functest
08:49:01 <ollivier> But we have to clarify which is in charge of pathes/tests. We don't have any resource and your patch is in our tree.
08:49:18 <morgan_orange> #info internship on K8 testing: welcome to Konrad, he will be mentored by Linda, Jose and Narinder
08:49:25 <morgan_orange> #info internship on Test API web portal: welcome to Thuvarakan he will be mentored by Serena
08:49:31 <morgan_orange> I will note the arm discussions in AoB
08:51:08 <ollivier> But I understand it's a huge work to maintain a full arch. But we can't block Functest regarding files we are not in charge of.
08:51:17 <ollivier> we will remove the former Dockerfile soon.
08:51:31 <ollivier> Welcome!
08:52:02 <morgan_orange> #info it would have been great that armband adopt alpines containers for E release. Maintaining 2 dockers is painful. Functets should probably have contacted armband earlier..initially it was a try and as it was working well it has been adopted
08:52:38 <morgan_orange> #info for the future we need to clarify which is in charge of pathes/tests. We don't have any resource and all arch related patches is in our tree.
08:52:50 <morgan_orange> #action morgan_orange initiate an etherpad to collect F features
08:53:29 <morgan_orange> #info refactoring of Docker file, split of Functest Framework / Functest-Test to be able to leverage the framework for Kubernetes and why not for Onap (exchange with ONAP community last week during ONAP dev days in Paris)
08:54:27 <CristinaPauna> #info alpine containers were pushed after ms5 and combined witht the dissagreement on how to integrate the arch in the image it made the adaptation of alpine to arm unreasonably short
08:54:57 <morgan_orange> #action morgan_orange ollivier prepare a presentation on Alpine / requirements management (slot booked on the testing group first) then possible to share with TSC
08:56:09 <morgan_orange> #info we need a consensus on the way to integrate arch image (with other test projects) and we need to follow what is already done and adopted by Open Source community (we did not discover the issues)
08:56:42 <morgan_orange> #info as already shared with the team, morgan_orange will move to ONAP after the release E, I would keep an eye for the review but will not be able to mentor and be assigned for F release
08:57:45 <morgan_orange> I saw that viktor_t info himself at the beginning, will youback in the game for F?
08:58:06 <viktor_t> Not probably yet...
08:59:45 <morgan_orange> BTW juhak the demos in Nokia (Paris) were fine :) no idea if you already went to Nokia Paris...it is not really in Paris..
09:00:29 <morgan_orange> ok if no more questions, I think we cna close the bridge, thanks to everybody including our Chinese contributors enjoying a full deserved break
09:00:41 <morgan_orange> #endmeeting