08:00:54 <jose_lausuch> #startmeeting Functest weekly meeting June 6th 2017 08:00:54 <collabot> Meeting started Tue Jun 6 08:00:54 2017 UTC. The chair is jose_lausuch. Information about MeetBot at http://wiki.debian.org/MeetBot. 08:00:54 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic. 08:00:54 <collabot> The meeting name has been set to 'functest_weekly_meeting_june_6th_2017' 08:00:56 <ollivier> It may work for testcases which don't require an external opnfv projects wich are not barometer, releng and snaps. 08:01:10 <jose_lausuch> #topic role call 08:01:14 <jose_lausuch> #info Jose Lausuch 08:01:15 <morgan_orange> #info Morgan 08:01:17 <ollivier> #info Cédric 08:01:25 <LindaWang> #info Linda Wang 08:01:45 <jose_lausuch> #info agenda for today: https://wiki.opnfv.org/display/functest/Functest+5.+Meeting#Functest5.Meeting-06/06(8UTC) 08:01:50 <juhak> #info Juha Kosonen 08:02:07 <CristinaPauna> #info Cristina Pauna (Enea) 08:02:15 <boucherv> #info Valentin Boucher 08:02:18 <jose_lausuch> the wiki is down now... 08:02:39 <OPNFV-Gerrit-Bot> Cedric Ollivier proposed functest: Use PrettyTable to show functest env https://gerrit.opnfv.org/gerrit/35807 08:02:40 <OPNFV-Gerrit-Bot> Cedric Ollivier proposed functest: Add the copyright header to decorators.py https://gerrit.opnfv.org/gerrit/35809 08:02:40 <OPNFV-Gerrit-Bot> Cedric Ollivier proposed functest: Remove a former action listed in framework.md https://gerrit.opnfv.org/gerrit/35811 08:02:41 <OPNFV-Gerrit-Bot> Cedric Ollivier proposed functest: Leverage on pbr (Python Build Reasonableness) https://gerrit.opnfv.org/gerrit/35813 08:03:23 <jose_lausuch> my company is blocking gerrit and the wiki 08:04:27 <jose_lausuch> #topic Action Point follow-up 08:04:31 <jose_lausuch> #info AP jose_lausuch: ask Steven to propose a session with Functest team to teach how to use snaps. DONE 08:04:37 <Shuya_ool> #info Shuya(OOL) 08:04:37 <jose_lausuch> #info the time is still to be decided by Functest team and Steven. 08:04:43 <jose_lausuch> #info AP all: review VNF class from morgan_orange. DONE 08:04:53 <jose_lausuch> #topic Summit sync 08:05:00 <jose_lausuch> #link https://wiki.opnfv.org/display/functest/2017+Beijing 08:05:08 <morgan_orange> note the patch has not been merged 08:05:10 <jose_lausuch> I Created a nice table with all the sessions from the functest memebers 08:05:25 <jose_lausuch> who can access the wiki? 08:05:31 <jose_lausuch> morgan_orange: ok, but it should be ready soon 08:05:42 <LindaWang> I can access the wiki. 08:05:59 <morgan_orange> hope so but leaving for beijing this afternoon - ONAP meeting as warmup for OPNFV summit this week... 08:06:29 <jose_lausuch> ok 08:06:44 <jose_lausuch> let's try to have it merged today if there are nothing against it 08:06:52 <jose_lausuch> if gerrit works for me.. 08:06:54 <morgan_orange> thanks for the table 08:06:57 <jose_lausuch> it works on my phone 08:07:04 <jose_lausuch> by don't want to use my phone for gerrit/wiki :p 08:07:33 <jose_lausuch> #link http://events.linuxfoundation.org/events/opnfv-summit/program/schedule 08:07:38 <jose_lausuch> #link https://wiki.opnfv.org/display/EVNT/Beijing+Summit+Idea+Nest 08:07:45 <jose_lausuch> SerenaFeng: are you ok with your session about the api ? 08:08:00 <jose_lausuch> I though it would be a good idea and I added it there 08:08:05 <morgan_orange> I think it will make sense to share also the presentation on the framework done by ollivier during a breakout session for people who would like to create their test cases 08:08:23 <jose_lausuch> I can do that in my session about hands-on 08:08:36 <jose_lausuch> I wanted to ask ollivier if he wants to be at the stage with me :) 08:09:20 <morgan_orange> it would be cool if we could add the vnf part..if we have time. But the current pres is already very clear for test case/feature projects 08:09:33 <jose_lausuch> this session http://linux-iframe.azurewebsites.net/SessionDetail.aspx?id=112300 08:09:41 <jose_lausuch> we can add it 08:09:45 <ollivier> Maybe you already integrate it into "Hands-On Testing: How to Integrate Tests in OPNFV" 08:10:01 <jose_lausuch> ollivier: do you want to come to the stage? 08:10:02 <morgan_orange> I also add SerenaFeng for the monday session, she will be the moderator for the panel 08:10:31 <jose_lausuch> morgan_orange: ok, but they should change the name in the session then 08:10:47 <ollivier> Let's discuss offline. I don't know what you have planed. I just see the title of your session and it's quite late 08:11:03 <jose_lausuch> ollivier: yes, let's talk later. No pressure! :) 08:11:08 <jose_lausuch> ok 08:11:13 <jose_lausuch> any question about the schedule? 08:11:21 <jose_lausuch> am I missing any session? 08:11:45 <ollivier> It will be great to discuss together (functest dev) on docker slicing and the related stuff. 08:11:46 <SerenaFeng> jose_lausuch, I updated the slides, I think I have shared with you and Morgan, but not sure whether you have received it or not 08:11:48 <morgan_orange> I think there is a breakout session to speak about functest roadmap/ Euphrates 08:12:06 <jose_lausuch> yes, I have the slides, thanks. I will review them soon 08:12:20 <jose_lausuch> but if they are similar to the slides we used for the plugfest they sohuld be ok 08:12:44 <morgan_orange> 13th 4-5 PM for the breakout sessions 08:12:54 <morgan_orange> I add the session to teh Functest page 08:13:10 <jose_lausuch> thanks morgan_orange 08:13:11 <SerenaFeng> sorry I lost some page. I can access to wiki. any help? 08:13:50 <jose_lausuch> SerenaFeng: what page? 08:14:18 <SerenaFeng> I mean the discussion before I came 08:14:31 <SerenaFeng> [16:05:17] <jose_lausuch> I Created a nice table with all the sessions from the functest memebers 08:14:31 <SerenaFeng> [16:05:32] <jose_lausuch> who can access the wiki? 08:14:44 <jose_lausuch> SerenaFeng: I mean this table https://wiki.opnfv.org/display/functest/2017+Beijing 08:15:24 <SerenaFeng> let's continue, we can discuss it offline, sorry for my interrupt 08:15:38 <jose_lausuch> SerenaFeng: no problem.It's just about if the info there is correct 08:15:45 <jose_lausuch> and created that table for our team 08:15:55 <jose_lausuch> to have fast visibility of who is presenting what 08:15:59 <SerenaFeng> okey 08:16:25 <jose_lausuch> #topic CI status (Master&Danube) 08:17:01 <jose_lausuch> #info Fuel Master: all master scenarios have been stopped. Only nosdn-nofeature is running due to the MCP integration by Fuel team 08:17:42 <jose_lausuch> #info Fuel Danube: looks good. Some scenarios have some little problems (old-l3, bgpvpn) 08:17:57 <jose_lausuch> for apex, I don't know much 08:18:28 <jose_lausuch> the ci structure has changed a bit 08:18:36 <jose_lausuch> I'm not sure how to judge this https://build.opnfv.org/ci/view/apex/job/apex-daily-danube/ 08:18:56 <jose_lausuch> or this https://build.opnfv.org/ci/job/functest-apex-baremetal-daily-danube/ 08:19:33 <jose_lausuch> there are some problems with Functest here 08:20:10 <LindaWang> Moreover, functest docker build failed due to the low version of setuptools . 08:20:23 <jose_lausuch> LindaWang: that's for master only 08:20:26 <jose_lausuch> but right 08:20:36 <jose_lausuch> let's discuss that in the AoB 08:20:41 <jose_lausuch> what setuptools version to use 08:20:47 <LindaWang> jose_lausuch: sure 08:20:50 <jose_lausuch> can you give status for compass? 08:21:02 <morgan_orange> on danube on reporting page, I do not see any results anymore for apex and joid 08:21:21 <jose_lausuch> master or danube? 08:21:28 <morgan_orange> I can see on fuel (x86 and aarch64) and compass 08:22:06 <jose_lausuch> you mean here? http://testresults.opnfv.org/reporting2/reporting/index.html 08:22:23 <morgan_orange> yes 08:23:16 <jose_lausuch> LindaWang: status for compass? 08:23:17 <morgan_orange> there is no more Functest apex job for Danube in jenkins 08:24:07 <jose_lausuch> the jenkins page is down for me now 08:24:10 <jose_lausuch> I can't check .... 08:24:20 <LindaWang> Sorry, I have not checked it. 08:24:41 <morgan_orange> correction.. 08:24:55 <jose_lausuch> morgan_orange: there is, I saw it 08:24:59 <jose_lausuch> but everything is red 08:25:43 <LindaWang> For master, snaps_smoke failed, but cause the functest image is not the latest, so it is hard to decide its status. 08:26:11 <jose_lausuch> ya, it's that master is not stable. I think it's not stable in installers as well 08:26:34 <LindaWang> But I tested locally on compass, there is something wrong with snapshot creating when running vping. 08:26:46 <LindaWang> So some bugs exit. 08:26:49 <jose_lausuch> ok 08:26:50 <morgan_orange> #info Danube on reporting page: few results for all the installers 08:26:54 <ollivier> Master was stable during all framework refactoring. Why is it broken now? It will be great to also consider switch to alpine 08:27:02 <morgan_orange> #info Master: unstable on installer and functest 08:27:42 <jose_lausuch> #info problem with building latest docker image due to bug in openstack-bgpvpn 08:27:43 <LindaWang> So I think it is urgent to fix the docker build job in CI. 08:27:46 <morgan_orange> #info Master CI results on fuel and compass only 08:27:49 <SerenaFeng> can we exclude these four test cases in snaps_smoke which always fail due to security group issue? 08:27:51 <jose_lausuch> #info workaround: upgrade setuptools in Functest 08:28:06 <jose_lausuch> #info fix merged on stable/newton. To be cherry picked to stable/mitaka 08:28:10 <SerenaFeng> and re-include them when they are fixed 08:28:12 <ollivier> Why have the change have been merged? 08:28:26 <jose_lausuch> ollivier: which change? the one in openstack? 08:28:39 <ollivier> "snaps_smoke which always fail due to security group issue" 08:29:12 <LindaWang> ollivier: Vping also failed when I tested locally. 08:29:24 <jose_lausuch> we can disable that test until we get a fix 08:29:26 <ollivier> So it should have been merged 08:29:36 <ollivier> havent't 08:30:01 <jose_lausuch> I don't get what change you mean 08:30:09 <jose_lausuch> snaps_smoke is running since danube 08:30:29 <morgan_orange> the problem is due to installer evolution..tests have not been modified 08:30:34 <morgan_orange> do we have a precise view on the installer status...as the same tests seem OK on some and not and others.. 08:31:12 <jose_lausuch> I don't have a view on that, I guess we will know soon what is the status, there is a milestone about openstack integration 08:31:20 <SerenaFeng> as I checked, snaps_smoke always faii in nosdn_nofeature scenario in all installers 08:31:26 <morgan_orange> on compass for example, on Master is it still Newton based? 08:31:41 <ollivier> morgan_orange: thank you to clarify. Then we should fix it soon. 08:31:49 <LindaWang> Is there some update in snaps. They passed before. 08:32:10 <jose_lausuch> can we file that as a bug in snaps jira? 08:32:17 <morgan_orange> but if SerenaFeng saw that is is failing on all the installers...it could be also due to a change in snaps that is not under our control... 08:32:19 <jose_lausuch> LindaWang: can you do that please? 08:32:23 <jose_lausuch> I can talk to 08:32:27 <jose_lausuch> Steven, to look into that 08:32:46 <SerenaFeng> I already send an email and discuss with him 08:32:47 <LindaWang> jose_lausuch: Ok, I will create a jira ticket for that. 08:33:01 <SerenaFeng> he said we can create a jira for him 08:33:05 <jose_lausuch> SerenaFeng: ok, I think I remember something, what is the conclusion? 08:33:26 <SerenaFeng> I will create a jira for him soon after the meeting 08:33:42 <LindaWang> SerenaFeng: ok. 08:34:12 <jose_lausuch> ok, thanks 08:34:29 <jose_lausuch> it's ok if we don't disable that test for now, as everything is not super stable 08:34:48 <jose_lausuch> if it still fails by test case ready milestone, we can disable it until it's fixed 08:34:54 <ollivier> I would propose to disable it 08:34:56 <morgan_orange> LindaWang: vPing is failling on compass/Master? - no link with SNAPS, and as far as i can see it is Ok on Fuel 08:35:13 <LindaWang> morgan_orange: I know. 08:35:36 <SerenaFeng> is vping's problem caused by hugepage? 08:35:44 <LindaWang> The issue is the docker image is not latest. 08:36:03 <morgan_orange> ok so several problems... 08:36:08 <SerenaFeng> Yes, daisy also fail due to image is not the latest 08:36:08 <LindaWang> SerenaFeng: No, some bug caused by frame refactor. 08:36:22 <LindaWang> minor error I think 08:36:38 <jose_lausuch> SerenaFeng: the "===" proble is fixed on newton, will it be cherry picked to mitaka? 08:36:41 <SerenaFeng> so several bugfix are not included 08:37:23 <LindaWang> Yes, so I believe if it makes sense to fix our docker build job in CI. 08:37:29 <SerenaFeng> I asked, but with no response 08:37:35 <ollivier> I'm not sure it will accepted as it concerns OpenStack oldstable. 08:37:43 <jose_lausuch> SerenaFeng: maybe you can just cherry pick it and add the reviewers... 08:38:07 <SerenaFeng> we can fix it by upgrading setuptools. we don't need to wait for that 08:38:09 <morgan_orange> so my view: priorities 1)new latest docker / 2) disable unstable cases / 3) reintroduce case once they have been successfully tested on at least one community lab? 08:38:09 <jose_lausuch> ok, so, regarding setuptools version? what do we do? fixed version or latest? 08:38:27 <ollivier> SerenaFeng: I think everything is precised in comments and I sent you an email to help cherry-picking 08:38:55 <SerenaFeng> ollivier, Great, thanks for your help 08:38:56 <ollivier> 4/ the most important is top fix the requirements issues 08:39:23 <LindaWang> I think 30.0.0 or 35.0.2 are both OK. But 30.0.0 is much better. 08:39:39 <jose_lausuch> #topic setup tools version to use 08:39:43 <jose_lausuch> let's have a topic for that 08:39:51 <SerenaFeng> I don't think specify one certain version is a good idea 08:39:55 <jose_lausuch> SerenaFeng: you said it's better to use latest, why? 08:40:13 <SerenaFeng> not only using latest, but a range of versions 08:40:36 <jose_lausuch> ollivier: any comment on the version to be installed? 08:40:41 <SerenaFeng> instead of one specified version 08:40:47 <LindaWang> We can not make sure that the latest of setuptools works or not. 08:41:00 <ollivier> The issue is bigger than only setuptools version. We don't manage any upper contrains, we erase requirements from a third party project to another and our setup.py is false. 08:41:17 <ollivier> We should sync with OpenStack version 08:42:06 <jose_lausuch> ollivier: that's right, but we can try to agree first on the setuptools version and fix what you say later 08:42:15 <jose_lausuch> since we need to have docker build working asap 08:42:32 <SerenaFeng> as we look into openstack project. they barely use one version 08:42:36 <jose_lausuch> what is opensatck ocata using for setuptools? 08:42:40 <ollivier> Yes. But we don't discuss about real issue. The most important is to fix this issue by using one of the correct version 08:42:46 <SerenaFeng> most of them provide a range of usable versions 08:43:26 <ollivier> But the real work is about stopping clone repo and pip install all requirements... 08:44:12 <jose_lausuch> https://github.com/openstack/nova/blob/master/requirements.txt#L34 08:44:38 <ollivier> that's perfect. 08:44:52 <jose_lausuch> we can do the same 08:45:40 <LindaWang> If we sync with OpenStack version, then most of others will also be fixed. Right? 08:46:22 <SerenaFeng> and if we using that, we should also exclude 36.0.0 08:46:30 <ollivier> Yes but we don't manage use OpenStack upper constraints. And as we are erasing everytime we don't know the result at the end. 08:46:31 <SerenaFeng> there are some problems with it 08:47:13 <jose_lausuch> ollivier: yes, we can stop this step by step, we need to study what we are erasing 08:47:27 <jose_lausuch> for example, I know bgpvpn requirements are installed after ours 08:47:32 <ollivier> jose_lausuch: simply by our current design 08:48:04 <SerenaFeng> but as regards to common tools like setuptools, most of project will not list it in requirements.txt 08:48:16 <jose_lausuch> #info agreed: the idea is to remove all the requirements installation after functest-requirements. 08:48:18 <ollivier> Please see the first change of the list: https://gerrit.opnfv.org/gerrit/#/c/35813/ 08:48:41 <SerenaFeng> we can fix this problem fist. and consider the erasing problem later 08:49:03 <jose_lausuch> so, do we agree on doing the same as https://github.com/openstack/nova/blob/master/requirements.txt#L34 ? 08:49:06 <LindaWang> SerenaFeng: agree. 08:49:19 <ollivier> I don't understand why we are debating for a false problem and we could have selected any stable version to fix our build issue. 08:49:23 <SerenaFeng> #agree 08:49:43 <jose_lausuch> ollivier: Linda selected a stable version already 08:49:53 <SerenaFeng> and don't forget to exclude 36.0.0, maybe also 36.0.1 08:50:01 <jose_lausuch> but it's not agreed by Serena 08:50:04 <ollivier> +2 08:50:08 <jose_lausuch> and I don't know what the best option is 08:50:13 <jose_lausuch> +2 on what? :) 08:50:18 <ollivier> I would heve slected the same rule as OpenStack but it can be merged 08:50:32 <ollivier> https://gerrit.opnfv.org/gerrit/#/c/35703/ 08:51:26 <SerenaFeng> Sorry, I just want to solve problem once and for all if there is better solution 08:51:32 <SerenaFeng> so I give -1 08:51:54 <jose_lausuch> I'm not an expert, but my logic says that we shouldn't use a "latest" version of a tool ... when the tool is upgraded, we cannot know if it works 08:52:03 <SerenaFeng> I don't want to introduce other problems considering various users 08:52:04 <ollivier> Don't worry. You're right to vote -1 here but we should focus on real issues. 08:52:54 <ollivier> So please review https://gerrit.opnfv.org/gerrit/#/c/35813/ and call docker run -t -i ollivier/functest-alpine /bin/sh 08:52:54 <ollivier> :) 08:53:02 <jose_lausuch> what do we do with 35703 then? 08:53:36 <ollivier> merge it to allow building the container and then we have time to sync with openstack 08:53:37 <SerenaFeng> update according to the link you provide 08:53:51 <jose_lausuch> hehe there is no consensus 08:54:25 <jose_lausuch> merge as it is or setuptools!=24.0.0,!=34.0.0,!=34.0.1,!=34.0.2,!=34.0.3,!=34.1.0,!=34.1.1,!=34.2.0,!=34.3.0,!=34.3.1,!=34.3.2,>=16.0 ? 08:54:28 <ollivier> jose_lausuch: Ofc there is a consensus. You have merged some patches by yourself to fix gate 08:54:59 <SerenaFeng> I will not fight for it. you can not merge this patch, and sync with OpenStack 08:55:08 <SerenaFeng> sorry 08:55:12 <SerenaFeng> you can merge this patch 08:55:16 <SerenaFeng> typo 08:55:39 <jose_lausuch> ok 08:55:55 <OPNFV-Gerrit-Bot> Merged functest: Update setuptools version to 35.0.2 https://gerrit.opnfv.org/gerrit/35703 08:55:56 <jose_lausuch> done 08:55:58 <jose_lausuch> merged 08:56:14 <jose_lausuch> let's see next docker build 08:56:37 <jose_lausuch> we should also sync all the openstack clients with opensatck ocata 08:56:38 <LindaWang> jose_lausuch: OK 08:56:45 <LindaWang> I will check it. 08:56:51 <jose_lausuch> we might have some that are outdated or something 08:57:11 <jose_lausuch> #topic AoB 08:57:21 <jose_lausuch> #info Functest REST API proposed by Helen: https://wiki.opnfv.org/display/functest/Functest+REST+API 08:57:32 <jose_lausuch> please have a look at it and provide feedback/comments 08:57:39 <LindaWang> Helen asked for a leave today. 08:57:44 <jose_lausuch> I know 08:57:54 <jose_lausuch> we can discuss this api during the sumit 08:58:03 <jose_lausuch> I'm afraid 1 hour for breakout is not enough... 08:58:06 <LindaWang> But Helen and I will go to the summit and we can discuss face to face maybe. 08:58:10 <morgan_orange> also for the summit 08:58:18 <morgan_orange> #info first test on alpine by ollivier https://hub.docker.com/r/ollivier/functest-alpine/ 08:58:46 <jose_lausuch> it would be great if everyone who has time to test the proposal from Cedric and provide feedback 08:58:51 <jose_lausuch> we save some space 08:58:54 <jose_lausuch> which is great 08:59:25 <LindaWang> I will help test it locally. 08:59:41 <ollivier> and it mainly manage correctly requirements.txt as everything is installed by one command 08:59:41 <LindaWang> ollivier: good job. 08:59:51 <morgan_orange> during the summit I did not see a session to share our view, especially on requirement management, it would be interesting to have a discussion with the other test projects and the infra group.. 09:00:20 <ollivier> Badly. It's a little bit early and I must beta test first 09:00:38 <ollivier> All setup.py (functest, releng,..) are wrong 09:00:39 <jose_lausuch> it's the time to test these things in CI 09:01:13 <morgan_orange> docker.io/ollivier/functest-alpine latest 9a713974520e 23 hours ago 223.1 MB 09:01:13 <morgan_orange> docker.io/opnfv/functest danube.1.0 6375e3f3ebfe 9 weeks ago 1.541 GB 09:01:20 <jose_lausuch> morgan_orange: in functest, we can decide what to do regarding requirements 09:01:31 <ollivier> The main point to discuss with releng is do we install releng as a python project or do we continue to copy shell 09:01:36 <jose_lausuch> we own the framework and must have the control about what's installed 09:01:44 <morgan_orange> +2 09:01:53 <ollivier> As internally it already raised lots of issues I may wait a little bit 09:02:02 <ollivier> see setup.py name. 09:02:31 <jose_lausuch> ollivier: I would like to wait until releng repo is split into 2 or 3 repos 09:03:01 <jose_lausuch> ok 09:03:04 <jose_lausuch> we are out of time 09:03:12 <ollivier> Yes. I understood. 09:03:17 <jose_lausuch> let's continue everything offline and on gerrit 09:03:18 <jose_lausuch> thank you all 09:03:21 <morgan_orange> ollivier: do you have access to gerrit/wiki from orange? 09:03:27 <jose_lausuch> and see you next week in China! 09:03:30 <jose_lausuch> #endmeeting