08:00:21 #startmeeting Functest weekly meeting March 28th 2017 08:00:21 Meeting started Tue Mar 28 08:00:21 2017 UTC. The chair is jose_lausuch. Information about MeetBot at http://wiki.debian.org/MeetBot. 08:00:21 Useful Commands: #action #agreed #help #info #idea #link #topic. 08:00:21 The meeting name has been set to 'functest_weekly_meeting_march_28th_2017' 08:00:26 Hi everyone 08:00:30 #topic role call 08:00:37 #info Morgan Richomme 08:00:42 #info Jose Lausuch 08:00:47 #chair morgan_orange 08:00:47 Current chairs: jose_lausuch morgan_orange 08:01:21 #info Juha Kosonen 08:01:42 #info Linda Wang 08:01:59 #info Helen Yao 08:02:06 #info Valentin Boucher 08:03:54 ok 08:03:58 #topic Action point follow-up 08:04:03 #info AP ollivier add a wiki page about review best practices. DONE https://wiki.opnfv.org/display/functest/Review+checkpoints 08:04:13 ollivier: thanks about it 08:04:29 #action everyone to agree on Cedric's proposal: https://wiki.opnfv.org/display/functest/Review+checkpoints 08:04:44 #info morgan_orange jose_lausuch prepare release note. To be finised this week (before release) 08:04:54 #info jose_lausuch exclude tempest_defcore (we can keep the code as it can be usefull to have a second launcher in case) but not execute it. DONE by morgan https://gerrit.opnfv.org/gerrit/#/c/30735/ 08:05:04 #info LindaWang contact juraj to indicate to update functest docker or at least to change the refstack criteria. 08:05:10 LindaWang1: ? 08:05:21 about the testcase test_create_delete_image Cisco meets, 08:05:26 It seems as identity issue. Have not rsolved this. 08:05:43 #info Cédric Ollivier 08:05:45 refstack criteria has been set to 100% 08:05:51 jose_lausuch: you're welcome 08:06:02 jose_lausuch: what is the status for the release? any news? 08:06:14 morgan_orange: let's talk about that later 08:06:22 morgan_orange: I think it's to be decided today 08:06:27 ok 08:06:29 LindaWang1: ok thanks 08:06:35 #info HelenYao ollivier jose_lausuch morgan_orange juhak LindaWang boucherv take care for your danube 1.0 JIRA. EMAIL sent and topic added 08:06:44 #info morgan_orange update the figures of the guides. DONE 08:06:46 morgan_orange: right? 08:06:54 yes 08:07:02 #info jose_lausuch check on bgpvpn status. ONGOING. Problems are being addressed. Need to see if sdnvpn will target 1.0 or 2.0 08:07:16 #info LindaWang set Refstack criteria to 100%. DONE 08:07:24 #info morgan_orange jose_lausuch prepare short summary and share with Functest contributors before the 27th 08:07:28 morgan_orange: we missed this 08:07:32 #action morgan_orange jose_lausuch prepare short summary "presentation on Functest (introduction/vision) + framework evolution and interface with third party to be initiated" and share with Functest contributors 08:07:36 #info Jun Li 08:07:57 #info SerenaFeng 08:08:00 any comments on the APs? 08:08:29 btw, happy birthday morgan_orange :) 08:08:48 happy birthday! 08:08:54 morgan_orange: happy birthday 08:09:07 morgan_orange: happy birthday! 08:09:14 (I hope it's today!) 08:09:37 yes it is... 08:09:43 #info Cristina Pauna 08:09:57 Happy birthday Morgan 08:09:59 morgan_orange: I hope you enjoy your 24 years old :p 08:10:00 answer to meaning of life + 1 08:10:08 #topic Danube status 08:10:12 So Is it a day off? 08:10:26 morgan_orange invites for beers today ! 08:10:30 #info APEX 08:10:39 let's go through the status of each installer 08:10:53 #info scenarios that are getting stable and getting 100%: nosdn-nofeature, odl_l3-nofeature, nosdn-kvm 08:11:02 #info scenarios that have problems with tempest_smoke: gluon, old_l3-fdio, odl-bgpvpn 08:11:11 #info scenarios that don't pass healtcheck: nosdn-fdio 08:11:27 anything else for apex? 08:12:04 I saw a patch for gluon and bgpvpn to exclude a vping (buf referenced) 08:12:09 yes 08:12:10 it was merged this morning 08:12:15 Merged functest: Switch TestcaseBase to TestCase https://gerrit.opnfv.org/gerrit/31871 08:12:22 Merged functest: Rename testcase_base to testcase https://gerrit.opnfv.org/gerrit/31879 08:12:22 #info patch for gluon and bgpvpn to exclude a vping_ssh in apex 08:12:28 Merged functest: Add docstrings in testcase.py https://gerrit.opnfv.org/gerrit/31907 08:12:44 #action jose_lausuch update release note with referenced bug for vping (gluon, bgpvpn) 08:13:02 morgan_orange: I'm noting down the things for the release notes 08:13:20 I juts put it here for reminder.. :) 08:13:25 I will publish a patch today hopefully, but after we have clarity about the release date 08:13:30 #info COMPASS 08:13:35 #info scenarios that are are fully working: nosdn-nofeature, nosdn-openo 08:13:39 #info scenarios that have problems with tempest_smoke: odl_l3-nofeature, odl_l2-nofeature 08:13:42 #info scenarios that don't pass healtcheck: ocl-nofeature 08:13:51 just by looking at the dashboard 08:14:15 for the ocl scenario, does anyone know which versions are used 08:14:24 it is also Newton? 08:14:43 maybe HelenYao ? 08:15:18 ocl is liberty 08:15:31 ocl is not supported by compass long time ago 08:15:39 OK so it is normal, it will never work 08:15:47 snaps includes fields that have been introduced in mitaka 08:16:15 then, why is it executed in ci if not supported? 08:16:25 it used to be a bit better some time ago...(able to run tempest/rally) 08:16:47 I believed it was based on Newton...I assumed in danube all the OpenStack shall be newton 08:17:01 #info ocl is libery and it isn't supported by compass long time ago 08:17:10 but I can exclude all snaps_health_check + snaps_smoke 08:17:12 that was the assumption 08:17:35 so shall compass keep on running it? 08:17:46 if so we can exclude the snaps tests leading to issue 08:17:53 but the scoring will not high anyway 08:17:59 it is more a question for scenario owner... 08:18:11 morgan_orange: leave it for the owner 08:18:18 let's not change that at this stage without confirmation 08:18:21 #action morgan_orange find compas/ocl scenario owner and asks for teh status 08:18:27 ok, thanks 08:18:38 #info FUEL 08:18:55 #info scenarios that are are fully working: nosdn-nofeature, nosdn-ovs, nosdn-kvm, odl_l2-nofeature, odl_l3-nofeature-noha 08:19:01 #info scenarios that have problems with features: sfc, bgpvpn 08:19:12 #info scenarios that don't pass healtcheck: nosdn-kvm_ovs_dpdk_bar, nosdn-kvm_ovs_dpdk 08:19:45 #info fully working os-nosdn-kvm_ovs_dpdk-noha 08:19:53 we had some downtime in fuel during the weekend 08:19:57 deployments were failing 08:20:10 impossible we are working on stable danube .. :) 08:20:25 fuel team has been fixing this and on monday it was back to blue status, so no iterations in 2 days 08:20:38 "stable" :D 08:21:05 stable is an ambiguous word.. 08:21:11 as stable as Ubuntu 08:21:16 lol 08:21:26 any more words on fuel? 08:21:48 ah! 08:22:12 #info excluded server_basic_ops from OLD scenarios in fuel. Bug reported upstream. --> to be included in the release notes 08:22:22 and thats it 08:22:25 #info JOID 08:22:30 #info scenarios that are are fully working: nosdn-nofeature 08:22:33 #info scenarios that have problems: nosdn-lxd 08:22:37 #info scenarios that don't pass healtcheck: odl_l2-nofeature 08:23:06 lxd => rally issue 08:23:12 #info there was a discussion about excluding functest for k8 scenarios (Functest doesn't support container based deployments yet) 08:24:00 do we have any plan for k8 support 08:24:02 rally and refstack 08:24:10 HelenYao: it's in the plan for E-release 08:24:27 we need to sit down and think about a way to support kubernetes 08:24:28 juhak: as far as I understand it is on nova https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-danube/51/console so theoretically no adaptation needed for lxd 08:24:31 or composite VIMs 08:25:04 would be probably a hot topic for E as several installers are speaking about k8 08:25:23 yes, we need to support it, it's a high prio 08:25:42 #topic JIRA review 08:25:46 #link https://jira.opnfv.org/browse/FUNCTEST-762?jql=project%20%3D%20FUNCTEST%20AND%20issuetype%20in%20(Bug%2C%20Task)%20AND%20status%20%3D%20Open%20AND%20fixVersion%20%3D%20%22Danube%201.0%22 08:26:17 #action ALL: check JIRA issues and try to finish them or postpone them to Danube2.0 or Euphrates when makes sense 08:26:29 let's see if we can clean JIRA before 1.0 08:26:35 could you do that ? 08:26:41 sure 08:26:43 sure 08:26:54 I sent an email with more details 08:26:57 ok 08:27:12 about https://jira.opnfv.org/browse/FUNCTEST-749?filter=-1 08:27:14 but there are not a lot of issues opened for Danube 1.0, I think we can make it 08:27:25 https://jira.opnfv.org/browse/FUNCTEST-762 must be clean soon 08:27:27 I have already submitted a patch 08:27:28 I do not know what to do with Prakash JIRAs, I suggest to postpone them to Euphrates... 08:27:35 any objection? 08:27:46 other option is delete them 08:27:55 SerenaFeng: I posted a comment about nested functions, not sure I understand why is that 08:28:13 LindaWang1: juhak: ollivier: morgan_orange, can you also review https://gerrit.opnfv.org/gerrit/#/c/30123/ ? 08:28:16 ok, I will look into it 08:28:20 morgan_orange: yes, let's postpone them 08:28:22 sure 08:28:23 soon after the meeting 08:28:40 ollivier: ok, thanks 08:28:41 yes 08:28:43 OK I do (even if deleting would make sense...not sure to understand why it is a JIRA) 08:28:51 morgan_orange: which one? 08:29:00 https://jira.opnfv.org/browse/FUNCTEST-616 08:29:07 https://jira.opnfv.org/browse/FUNCTEST-614 08:29:18 I do not know what to do with them 08:30:11 so delete... 08:30:12 jose_laushuch: I fixed "push to db error" in vrouter module. I will confirm to weekly CI result. After confirm, I will close JIRA. 08:30:17 morgan_orange: let me delete them 08:30:22 ok 08:30:41 -2 08:30:49 https://jira.opnfv.org/browse/RELENG-185 08:30:58 I submitted a patch long time ago 08:31:06 but no further process 08:31:13 should I abandon it or renew it? 08:31:17 https://gerrit.opnfv.org/gerrit/#/c/26501/ 08:31:27 mmmm 08:31:45 sorry about it... i missed it 08:31:48 shall we postpone it for E ? 08:32:03 or 2.0 ? 08:32:20 how about 2.0? I will renew it right away 08:32:39 rebase will be hard ... :) 08:32:43 as you wish 08:32:49 I think it makes sense to use releng constants if we have an adoption by the testing community 08:33:01 if only functest does it...it could be in functest 08:33:19 maybe we could have a topic during the next test working group APAC meeting to present that for E? 08:33:27 sure 08:33:43 #action propose https://jira.opnfv.org/browse/RELENG-185 to the APAC group test WG 08:33:46 #undo 08:33:46 Removing item from minutes: 08:33:51 agree 08:33:52 #action morgan_orange propose https://jira.opnfv.org/browse/RELENG-185 to the APAC group test WG 08:34:07 it could even be a testing priority (common exception classes, consts, ..) 08:34:12 are you ok with that SerenaFeng 08:34:14 we made lots of efforts for unifying, but for now, only functest are using them 08:34:28 SerenaFeng: we need more advertising :) 08:34:51 and ollivier is preparing pylint + embeeded documentation ... 08:35:04 morgan_orange sorry I don't get your point, agree with what? 08:35:09 we are trying to share our best practices 08:35:27 postponed to E? or using releng constants? 08:35:42 be careful about unfying. TestCase defines internal return codes which could differ from those defined in releng (external) 08:35:44 make a presentation during a testing working group in order to adopt it in E ion the different test projects 08:36:03 it could also be discussed during the summit... 08:36:08 ollivier: we could make them the same as we are initiating that 08:36:09 we are mixing that... 08:36:09 agree, that would be great 08:36:19 unifying takes time (see adoption of the DB) but step by step we do it... 08:36:31 Ok I will paln something for the next APAC slot 08:36:33 morgan_orange: I think it's also a good time to advertise these common tools during the Hackfest 08:36:53 They could be the same values and meanings but releng status should be used only by run_tests.py 08:36:53 we could have a short demo showing the tools in releng... 08:37:14 we should advertise the work in releng/modules a lot 08:37:16 All testcases should use the constants defined in testcase 08:37:19 #action propose demo for the hackfest about the available tooling in releng 08:37:40 again... 08:37:41 #undo 08:37:41 Removing item from minutes: 08:37:48 #action jose_lausuch propose demo for the hackfest about the available tooling in releng 08:37:57 And we should update them to use Enum 08:38:14 ollivier: can you and SerenaFeng work together on that? 08:38:17 + pay attention to Cedric remarks => we should also not mix the return codes...so need some time, relax breath and consider that for E :) 08:38:33 ya 08:38:39 I think it's not feasible for 2.0 maybe 08:38:49 we don't need to rush 08:38:57 +1 08:39:01 jose_lausuch, ok, I can work on that with ollivier 08:39:11 we can just start it now and leave it in master only 08:39:15 sure. 08:39:22 as it's the purpose of the master branch 08:39:30 so we are not stuck with danube branch :) 08:39:36 ok 08:39:36 #topic OPNFV summit proposals 08:39:40 #info the CFP deadline has been extended to 4/10 (2 weeks more) 08:39:45 #link https://etherpad.opnfv.org/p/TestCommunitySummit 08:40:02 these are the topics we will have 08:40:06 #info TOPIC: Design Session / Morgan (panel) 08:40:12 #info TOPIC: hands on testing / Design Summit / tutorial / Jose 08:40:20 #info TOPIC: Upstream and Xcollaboration (refstack, odl-netvirt) / Jose 08:40:24 #info TOPIC: challenge / micro service approach how to consumme API /Design summit / Morgan 08:40:27 #info TOPIC: Testing and CI gating / Design Summit / Nikolas 08:40:37 I still need to submit mines 08:40:45 last one is already submited 08:40:49 do I miss any? 08:40:52 morgan_orange: ? 08:40:58 TestAPI Enhancement 08:41:03 ya! 08:41:16 #info TOPIC: TestAPI Enhancement / Design Summit 08:42:02 there is also a proposal for VNF catalogue from kumar 08:42:20 #info TOPIC: VNF catalogue 08:42:21 ok 08:42:31 I think it would make sense to speak about our abstractions / best practices 08:42:45 during the design summit 08:42:53 good idea 08:43:12 #action propose a new talk about abstractions / best practices / common libraries / ... 08:43:42 #topic Review checkpoints 08:43:45 #info Cedric initiated https://wiki.opnfv.org/display/functest/Review+checkpoints 08:44:00 yes it will be a part of this 08:44:00 This covers all the best practices when reviewing code 08:44:15 ollivier: you want to say something about it? 08:44:15 i assume Cedric could be the right one as the daddy of TestCaseBase 08:44:22 yes :) 08:44:34 not "base" any more... just Testcase :) 08:44:55 should I rename feature_base to base also? 08:45:04 or just feature 08:45:05 feature_base to feature 08:45:09 ya 08:45:12 yes typo 08:45:13 Sure. This proposal should help reviewing code and I would like everyone to agree. 08:45:14 sorry 08:45:15 ok 08:45:35 #action SerenaFeng rename feature_base to feature 08:46:01 ollivier: the idea is to be more strict in gerrit, at least starting in Euphrates 08:46:10 we need everyone to agree on that 08:46:22 shall we do a vote next week once eveyone has read it? 08:46:33 let's plan a vote next week 08:46:42 Ashish Kumar proposed functest: More Unit Tests for utils module https://gerrit.opnfv.org/gerrit/32009 08:46:48 you can add comments in the wiki as well 08:46:53 Yes. Then every committer must know them and apply them 08:47:11 not only committer 08:47:13 #action morgan_orange propose a vote for https://wiki.opnfv.org/display/functest/Review+checkpoints next week 08:47:16 It's time to use pylint now 08:47:18 (I will be at the ONS) 08:47:25 all the people submit patches to functest 08:47:45 #info ollivier will propose pylint gating for commits, replacing the existing flake8 check 08:47:51 At least committers 08:48:21 ok, thanks 08:48:27 #topic IMS abstraction 08:48:32 Pylint should use as help. We must continue on voting -1 if pep8 returns failures 08:48:32 morgan_orange: you added this topic 08:48:50 ollivier can you give us a brief introduction about pylint, and the differences with pep8? 08:48:55 We can accept several pylint warnings 08:49:08 Pep8 is just a subset of pylint 08:49:17 https://www.pylint.org/ 08:49:34 it has PEP8 + error detection 08:49:44 Please see https://gerrit.opnfv.org/gerrit/#/c/31915/ 08:49:45 you can install it on eclipse 08:50:15 ollivier: I saw it yetserday :) 08:50:19 Merged functest: Fix pylint warnings in odl https://gerrit.opnfv.org/gerrit/31915 08:50:21 It checks naming, docstings, try...except... unused args... 08:51:03 And finally it rated your scripts :) 08:51:05 morgan_orange: what about ims abstraction? 08:51:28 HelenYao: submitted a patch 08:51:48 must docstrings be provided? 08:51:49 I think it is OK but orchestra must be excluded as it is dealing with IMS (but with OpenIMS not clearwater IMS) 08:51:59 Yes, the patch is here https://gerrit.opnfv.org/gerrit/#/c/30681/ 08:52:09 Docstring is required in super class or external APIs 08:52:14 morgan_orange: I agree 08:52:20 so I would suggest, no change in orchestra it is up to orchestra and adopt the changes (cleawaterIms..) 08:52:24 ah ok that patch 08:52:25 I am not sure is relevant in test units 08:52:26 what name for the abstraction is appropriate 08:52:29 I think is fine for 2.0 08:52:32 lots of good improivements...pyling would have probably shout on us... :) 08:52:59 In a first step I would propose to disable docstring in uni tests... 08:53:10 the problem of this patch is that we shouldn't change 409 lines of code right before releasing... 08:53:28 the last one you suggested ClearwaterImsOnBoardingBase looks fine for me (it is explicit) 08:53:36 most of the codes are related to clearwater api calls 08:53:39 when do we release... 08:53:42 :) 08:53:58 Any patch should modify 409 line of codes... 08:54:07 the changes are big but affect only cloudify_imls and opera_ims (if we exclude orchestra) 08:54:24 and today, we can check only through 1 weekly job.. 08:54:30 ya 08:54:35 I do not see a big risk... 08:54:39 the orchestra is basically untouch if i revert it to inhert from vnf_base 08:54:48 BTW HelenYao I did not get any feedback on the creation of a weekly job on compass 08:54:56 it will be good to check weekly test suites 08:55:16 do we agree on excluding orchestra? 08:55:31 exclude from the patch...not from the release 08:55:37 ollivier: should or shouldn't ? 08:55:44 morgan_orange: yes, form the patch 08:55:50 morgan_orange: who should provide the feedback for compass weekly job? 08:56:00 shouldn't 08:56:01 the case is still not 100% OK but we can deploy OpenBaton, there is an issue deploying the VNF but it is hard to troubleshoot... 08:56:27 ollivier: agree 08:56:48 ollivier: keep small patches wich are dependent to each other if needed 08:57:08 jose_lausuch agree 08:57:10 let's take a close look at the patch, esp. this file 08:57:12 https://gerrit.opnfv.org/gerrit/#/c/30681/17/functest/opnfv_tests/vnf/ims/opera_ims.py 08:57:12 Sometimes it's hard as we want to propose a big function 08:57:19 big patch is difficult to review 08:57:23 this files includes all unnecessary lines 08:57:40 which causes 400+ line change 08:57:41 ollivier: yes, we should apply that when possible. Some refactors don't work with just a few lines 08:58:20 I would propose not to fill the file with unneccessary code 08:58:36 But such a big work must be merged. 08:58:43 HelenYao: also, as morgan_orange proposes to exclude orchertra 08:58:45 the opera_ims was not implemented while it has tons of code before this patch 08:59:17 the patch is fine.. 08:59:53 and teh change is not a change as explained by Helen it was a dummy file...the impelmentation get riff od the dummy code... 08:59:53 #topic AoB 09:00:09 morgan_orange: i will revert the openbaton and change the ims class name 09:00:22 morgan_orange: can you take next weeks' meeting ? I will be at the ONS summit in San Francisco 09:00:27 HelenYao: yep perfect 09:00:35 bad time for me 09:00:35 jose_lausuch: ok 09:00:43 (although i might be awake because of jet lag) 09:00:45 thanks 09:00:46 You have to rebase it. 09:00:48 #info Updated the Euphrates page: https://wiki.opnfv.org/display/functest/Functest+Euphrates+page 09:00:50 it will be vacation time...release will be over 09:00:58 morgan_orange: hopefully... 09:01:09 anything else? 09:01:14 we are over time, as usual :) 09:01:35 I will update the reporting page with the new icons provided by LF design team.. 09:01:50 I contacted bitergia, it is maybe time to reconnect and see how to proceed with them 09:01:50 ok 09:02:00 I saw that 09:02:04 we should 09:02:13 if they are going to the hackfest.. maybe good time to further discuss 09:02:16 test case catalogue page patch has been submitted..try to check it. If Ok I will push it to testresults 09:02:30 maybe we shoudl check if the description of our test cases is accurate...in the DB 09:02:30 morgan_orange: yes, go ahead 09:02:37 ollivier: sure 09:03:13 and if the release date is discussed this afternoon 09:03:18 what is your view on the topic? 09:03:26 on what topic 09:03:32 the status? 09:03:33 well 09:03:37 are we OK to release 09:03:46 I'd say that some scenarios have to wait for 2.0 since they are failing still 09:03:59 but yes, we can release... 09:04:07 I'll try to be on the call 09:04:19 let's discuss it offline 09:04:22 #endmeeting