08:02:09 #startmeeting Functest weekly meeting January 24th 2017 08:02:09 Meeting started Tue Jan 24 08:02:09 2017 UTC. The chair is jose_lausuch. Information about MeetBot at http://wiki.debian.org/MeetBot. 08:02:09 Useful Commands: #action #agreed #help #info #idea #link #topic. 08:02:09 The meeting name has been set to 'functest_weekly_meeting_january_24th_2017' 08:02:22 #topic role call 08:02:26 #info Jose Lausuch 08:02:29 #info Morgan Richomme 08:02:36 #info Helen Yao 08:02:40 #chair morgan_orange 08:02:40 Current chairs: jose_lausuch morgan_orange 08:02:50 #info SerenaFeng 08:02:50 #info Linda Wang 08:02:55 #link https://wiki.opnfv.org/display/functest/Functest+Meeting#FunctestMeeting-24/01(8UTC) 08:03:51 #topic Action point follow-up 5' (of the previous 2 meetings) 08:04:01 #link http://ircbot.wl.linuxfoundation.org/meetings/opnfv-functest/2017/opnfv-functest.2017-01-10-08.00.html 08:04:19 #info AP1: HelenYao check compass issues (issue with keystone version during snapshot/clean phase) 08:04:27 #topic action items 08:04:33 #undo 08:04:33 Removing item from minutes: 08:04:40 didnt see you put it 08:05:02 HelenYao: I think this AP is under control 08:05:17 the baremetal is failing and the virtual is ok 08:05:33 #info baremetal is failing and the virtual is ok 08:05:41 https://jira.opnfv.org/projects/FUNCTEST/issues/FUNCTEST-680 08:05:51 this epic has all the open issues on CI 08:05:51 #link https://jira.opnfv.org/projects/FUNCTEST/issues/FUNCTEST-680 08:06:08 #info AP2 morgan_orange initiate a mail with Functest contributor to indicate current status on community lab access 08:06:08 steve ask for access to ad oc community labs 08:06:33 #info done (got feedback on the mailing list)+ great virtual labs from jose 08:06:50 #info AP3 steve ask for access to ad oc community labs 08:06:52 #info done 08:07:00 #info AP4 jose_lausuch remove colorado view in jenkins functest page 08:07:02 #info done 08:07:09 #info AP5 jose_lausuch update JIRA sprint 08:07:13 you are too fast :) 08:07:13 I think it is done.. 08:07:17 #info done 08:07:26 this week we start sprint 5 08:07:40 #info sprint 5 starts this week 08:07:44 #info AP6 ashishk sinc with Mark Beier 08:08:14 is he in the chat? 08:08:24 #info mbeir made a presentation on unit test last week during testing wg meeting, new one is planned to exchange on the project best practices 08:08:41 #action ashishk sinc with Mark Beier 08:08:52 #info AP7 plan slot with GTM on the 26/1 for unit tests => done 08:09:02 about this 08:09:06 #info AP8 plan slot with GTM on the 2/2 for unit tests => done 08:09:15 at that time I will be on the train 08:09:39 there is no rush we may postpone if it is more convenient 08:09:46 please update the meeting page 08:09:53 so I will not attend it 08:09:55 all members in china will be OOO on Feb.2 08:10:08 official holiday? 08:10:12 yes 08:10:13 OK so makes sense to move that to the 9th 08:10:14 ok 08:10:17 yep 08:10:19 the last day of spring festival 08:10:31 Feb.9 sounds good 08:10:40 #action jose_lausuch review agenda for teh 2 (public Holiday in China) 08:10:49 #info AP9: morgan_orange plan a meeting tomorrow 5PM CET to discuss VNF on boarding - send invitation and bridge details 08:11:00 #info done, on VNF onboarding patch submitted last week 08:11:12 #info AP10 all feedback requested on https://gerrit.opnfv.org/gerrit/#/c/26695/ 08:11:21 #info done and merged 08:11:33 now teh AP from last week :) 08:11:45 #info AP11: jose_lausuch contact Copper PTL to see what happens with Copper test case 08:12:02 I think LindaWang is trying to fix path issues dealing with copper 08:12:05 #info Bryan was saying that he can't test it without community labs 08:12:27 possible to test on virtual pod (Apex), at least the path issue.. 08:12:48 can we action LindaWang to test her patch on the virtual pod for that? 08:12:53 yes 08:12:59 ok 08:13:13 #action LindaWang check path issue with copper on apex virtual pod 08:13:19 #info AP12: jose_lausuch contact Doctor PTL for Doctor test case 08:13:27 #info not done 08:13:47 #info AP13: juhak troubleshoot on Apex error 08:13:54 juhak: any update 08:14:00 #info occasional failures, seems to be related to odl 08:14:11 Tim's comment in APEX-380: "ODL failed due to a floating IP bug in ODL, which should be resolved by building a new RPM for ODL" 08:14:52 juhak: do we have the same behaviour in our vpod? 08:15:20 juhak: if we get a new RPM do you know how to re-deploy with that new package? 08:15:21 is there vpod deployed with odl? 08:15:36 juhak: no 08:15:42 juhak: maybe we could add it 08:15:52 something we already discuss and is beyond Functest scope, it would make sense at the beginning of the test to get the version of components i.e. odl, it would make sense to get this information from the installers 08:16:08 #info AP14 talk to trozet about openstack version vs keystone version. Can we use v3 as all the others? 08:16:33 #action jose_lausuch talk to trozet about openstack version vs keystone version. Can we use v3 as all the others? 08:16:45 #info AP15 review all https://gerrit.opnfv.org/gerrit/#/c/27015/ 08:17:01 #info merged 08:17:01 #info done and merged 08:17:05 man :D 08:17:18 #undo 08:17:18 Removing item from minutes: 08:17:20 #info AP16 SerenaFeng get an status update about Compass deployment problems 08:17:38 it was HelenYao not SerenaFeng because Zerena is working for Zte and Helen fir Huawei 08:17:47 yes 08:17:53 but I think it is duplicated anyway 08:17:55 yes ;) 08:17:58 there was already an AP 08:18:01 the compass has been working on the endpoint issue 08:18:13 once that patch is fixed, the baremetal build would move on 08:18:14 #info AP16 canceled => see AP17 08:18:15 #info AP 17 HelenYao get an status update about Compass deployment problems 08:18:43 yes we saw new results on the reporting page, including compass deployement with SNAPS tests OK... 08:18:58 #info AP18 jose_lausuch talk to jmorgan1 about intel pods status 08:19:14 #infopods seem back to life, new runs from joid seen in reporting page 08:19:15 #info intel pods are back, JOID is back 08:19:21 do not undo 08:19:32 it is better than my inputs.. :) 08:19:39 you missed a space this time :p 08:19:44 #info AP19 morgan_orange contact narinder to get visibility...would be hard to troubleshoot if no run before the end of january 08:20:01 #info see AP18, joid is back, issues with SNAPS see next topics 08:20:18 #info AP19 jose_lausuch test connection_check on a fuel env 08:20:39 #unfo 08:20:41 #undo 08:20:41 Removing item from minutes: 08:20:41 #info didn't have time myself, but Steven tried and found to root cause 08:20:56 #info AP20 jose_lausuch test connection_check on a fuel env 08:21:18 20 APs i think it is our record... 08:21:19 #info problem with the RC file in Fuel. Maybe we could tweak the RC file coming from the installer (fetch_os_creds) 08:21:35 probably similar issues with joid 08:21:47 ya 08:21:57 the v3 in the endpoint url was missin 08:21:59 strange 08:22:01 anyway 08:22:04 will continue with that 08:22:07 #topic Functest server 08:22:18 #info Baremetal server provided to Functest team. 08:22:33 #info 2 virtual deployments: Fuel + Apex 08:22:43 #info thanks to infra team! very appreciated 08:22:55 #info we should find a way to get similar resources for compass/joid 08:22:56 #info it is possible to create more deployments in Fuel so that each one of us have a different one to play with 08:23:21 hehe yes 08:23:29 I'm also very much an infra guy :) 08:23:42 jose_lausuch: bravo 08:23:57 #action create more than 1 fuel deployment. Resources are ok (enough disk and ram) 08:24:02 who of you want a fuel deployment? 08:24:09 I can try to create up to 3 or 4 08:24:14 I'd like to have 1 :) 08:24:32 you think it makes more sense to have a dedicated by tester or different scenario 08:24:42 HelenYao: LindaWang: is it ok for you to share one? 08:24:46 I think it would make sense to have a nosdn-nofeature, a odlènofeature 08:24:54 and 2 that could be changed depending on the need 08:24:56 yes 08:24:59 jose_lausuch: it's ok 08:25:02 jose_lausuch: I am fine with it. 08:25:05 bgpvpn, sfc, ovs, ... 08:25:18 I would like to go so deep 08:25:23 for example 08:25:28 there is a POD already for SFC guys 08:25:32 no need to do it ourselves 08:25:38 same for bgpvpn 08:25:45 I would add only ODL 08:25:55 well, I will add the plugin 08:26:07 and I'll assing a deployment for everyone 08:26:11 and then you can do whatever you want 08:26:22 you can re-deploy many times in the gui 08:26:31 you can activate odl and so on 08:26:35 would that be ok? 08:26:44 I think we can deploy them with different scenarios 08:27:00 once the fuel mode works, we can set up a compass by following the same pattern 08:27:04 so everyone can access 08:27:17 but what scenarios? 08:27:20 in this way every one will have resources to test 08:27:34 scenarios functest supported 08:27:36 my idea is that everyone gets a pod to play with 08:27:40 and no collisions 08:27:50 sdn /nosdn/ odl_l2/odl_l3 ...etc 08:28:14 I can prepare those 3 as well 08:28:14 ok 08:28:30 r u referring to deploying one scenario at a time on the pod and recreate it per everyone's interest 08:28:31 let's do it like that then 08:28:48 if all the pod are the same scenarios, still there will be some tests cannot be tested 08:29:18 #action create 3 scenarios: nosdn, odl_l2, odl_l3 08:29:51 ok? 08:29:56 ok 08:29:58 awesome 08:30:01 ok 08:30:25 ok 08:30:27 if there is anything that we can help, pls let us know 08:30:28 for Apex I can't do that, at least I think we can't have more than 1 at the same time.. 08:30:33 HelenYao: ok, thanks 08:30:50 what scenario do you want in apex? 08:30:53 nosdn as now? 08:31:47 we can wait to see if the nosdn is working. if it works, we can set up more 08:31:59 ok 08:32:02 good 08:32:12 #topic Troubleshooting status (short update) 08:32:29 #info morgan_orange and HelenYao already sent some information by email 08:33:00 #info JOID is back and running Functest https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-master/ 08:33:24 #info some ugly warnings in the healthcheck 08:33:46 #info There was a JIRA about refactoring the healthcheck, but I think we should close it and replace it by SNAPS asap (when it works on all the installers) 08:34:04 yep +1, wait for SNAPS to be OK on all installers then remove bash 08:34:36 +1 08:34:37 there is a small issue 08:34:45 healthcheck creates a flavor that vping uses 08:34:55 if you run vping without healhcheck it fails due to missing flavor 08:35:03 maybe we should add the creation of that flavor 08:35:04 #link http://testresults.opnfv.org/reporting/functest/release/master/index-status-apex.html 08:35:05 yeah, that is a problem 08:35:18 agredd 08:35:20 i think we need to create the flavor during the prepare_env 08:35:26 +1 08:35:29 morgan_orange: no weather icons? 08:35:30 vping shouldn't rely on healthcheck 08:35:41 HelenYao: I would like to avoid that 08:35:51 they are independent testcases 08:35:55 each test case should create whatever it needs 08:36:01 agree 08:36:02 we seem to use m1.tiny, m1.large in our code 08:36:11 and should clean the resources it created 08:36:18 SerenaFeng: yes 08:36:22 yes each test shall be autonomous 08:36:27 we also need to add flavor clean in openstack_clean 08:36:34 for now it is okey, because all the testcases run in serial 08:36:36 jose_lausuch: in that way, we need to add pre_env and clean in testcasebase 08:36:49 it will be a problem after we run them in parallel 08:36:51 pre_env? 08:36:58 yes 08:37:06 in the vnfBase, there is a prepare method to create user/tenant/..; 08:37:08 using SNAPS we don't need to do our cleanup 08:37:22 yeah, the testcasebase should provide abstract method of pre_env and clean_env 08:37:39 create tenant/user/flavor should be managed by testcase itself 08:37:52 each testcase need different resources 08:38:04 SerenaFeng: agree 08:38:04 it is difficult for testcasebase to manage them 08:38:13 for VNF onboarding we systematically need that 08:38:45 we create a tenant/user with the name of the vnf and we clean everything at the end 08:38:54 an abstract method in testcasebase and the child class can implement is based on its real need 08:39:19 I think we need to sync some work in featurebase to testcasebase 08:39:24 in that sense, that is very similar to HEAT 08:39:49 jose_lausuch: what is HEAT? the openstack service? 08:39:54 oh 08:40:05 like prepare/post/log_results....etc 08:40:09 HelenYao: HEAT is cool, it's a short way to create all the resources you need 08:40:16 HelenYao: it has its own cleanup 08:40:33 jose_lausuch: could u provide a link? 08:40:38 https://wiki.openstack.org/wiki/Heat 08:40:43 it will probably be a bit short to rething that for Danube 08:40:52 I think that the priority is now to make the test cases run 08:40:53 http://docs.openstack.org/developer/heat/template_guide/hot_guide.html 08:40:59 and see such evolution for E 08:41:02 morgan_orange: +1 08:41:11 +1 08:41:27 +1 08:41:29 ok 08:41:33 let's move on 08:41:36 for joid/SNAPS the error is strange return os_credentials.OSCreds(username=config['OS_USERNAME'], 08:41:36 KeyError: 'OS_USERNAME' 08:41:44 this env variable is in theory created 08:42:02 but when SNAPS fail, it exists and we do not have the functest.log to check 08:42:05 ya, it should 08:42:33 shall we ask Steven to modify that behaviour? 08:42:41 we can discuss with him 08:42:49 can you take the action? 08:42:55 but I think it makes sense to get a FAIL status without exiting the CI 08:43:01 OS_USERNAME is demanded for every auth 08:43:02 me too 08:43:10 #action morgan_orange contact steve to discuss exit conditions in SNAPS 08:43:21 thanks 08:43:26 how come OS_USERNAME is not provided 08:43:28 HelenYao: yes that is why is surprising...Tempest suite works well.. 08:43:39 without OS_USERNAME, nothing will work 08:43:40 hmm, interesting 08:43:44 but ok let's move on 08:43:46 maybe it is provided, but not part of config[] 08:43:48 #topic Dockerfile for ARM 08:44:11 #info ARMBAND team needs a different Dockerfile to run Functest on ARM based PODs 08:44:27 #info the differences are not big, but still it's a different Dockerfile 08:44:34 the question is 08:44:45 they willl provide a build -server to build the image 08:45:00 1) Should we update a new image with a new tag? latest_arm or something? 08:45:10 what is the difference? 08:45:13 2) How do we do the automated build in jenkins? 08:45:27 some x64 libraries we install need to be different 08:45:41 for example 08:45:47 we can create a new job for arm on CI 08:45:58 yes, thats clear 08:45:59 but 08:46:15 is it safe to have 2 Dockerfiles in the /docker directory? 08:46:16 like 08:46:18 Dockerfile 08:46:20 Dockerfile_arm 08:46:28 we can rename the dockerfile 08:46:31 I don't think two different docker image is a good idea 08:46:57 ok, this is where your input is very appreciated 08:46:58 ideas? :) 08:47:16 i think having 2 dockerfiles are safe 08:47:26 2 dockerfiles is not a problem 08:47:38 docker build support specify dockerfile 08:48:03 we can test how much is the package difference between arm and x86 08:48:05 another difference is : instead of "FROM ubuntu:14.04" it's "FROM aarch64/ubuntu:14.04" 08:48:31 okey, that definitely needs two dockerfile :) 08:48:40 if the original base image is totally different, I will vote for two dockerfile 08:49:00 is there any other option? 08:49:04 https://hastebin.com/etiyoxanec.diff 08:49:22 ok 08:49:37 jose_lausuch: cannot get your difff here (proxy) 08:49:39 #info We need 2 dockerfiles, our default one and another one for ARM 08:49:50 but the question will be the same for other projects 08:50:00 e.g. yardstick, qtip 08:50:07 the arm support will raise the same question 08:50:27 ollivier: http://pastebin.com/raw/NgheSrCv 08:50:41 arm might focus first on functest 08:50:43 then, we will see 08:50:45 we could also consider switching to alpine 08:51:06 you already had that good idea some time ago 08:51:06 i am a bit worried about the package support on alpine 08:51:15 but Helen did some investigation 08:51:15 jose_lausuch: pastebin is blocked too. I will switch to my public access :) 08:51:27 ollivier: too many restrictions in your proxy :) 08:51:42 alpine is like considering heat for resource preparation... a good topic for E now :) 08:51:48 yes 08:51:52 +1 08:51:53 too late for D 08:52:01 jose_lausuch: that's why I am always connected to my public net. 08:52:22 so 08:52:27 how do we trigger the docker build? 08:52:35 do we trigger both builds? 08:52:44 i think so 08:52:54 the default on any opnfv build server, and the other one at the same time on the ARM build server 08:52:56 just the same way as we do now for x86 08:53:12 ahd pushing the 2 images 08:53:15 what tag? 08:53:20 latest_arm ? 08:53:45 jose_lausuch: I will use a dedicated name opnfv/functest_arm 08:53:50 how about functest_arm? 08:53:54 aha 08:53:54 +1 08:53:55 ok 08:54:01 so, a new docker repo 08:54:24 but what happens when yardstick also needs to support arm 08:54:25 I am thinking the same, is it okay to have a new repo? 08:54:28 another repo for that? 08:54:29 yes. and you should keep the same tags 08:54:29 mmmm 08:54:34 not very much scalable 08:54:52 but clearer 08:55:00 yes 08:55:03 opnfv/yardstick_arm opnfv/storperf_arm ? 08:55:06 tag will confuse user 08:55:08 how about put the arm in one repo, for functest and yardstick? 08:55:11 I don't understand the issue regarding the scalability 08:55:31 if that is the case, it would be opnfv/arm:functest 08:55:40 i am not sure about this 08:55:50 we are duplicating the repos if arm wants to support all the test projects 08:56:17 if putting as opnfv/arm, it will be ok for all test projects 08:56:30 but more confused 08:56:32 yes 08:56:43 we have to balance 08:56:44 how do we now say colorado arm functest? :D 08:56:52 tag is used to manage version, not identify project 08:56:54 opnfv/arm:functest_danube ? 08:56:57 not clear 08:57:02 ya 08:57:08 I'd prefer different repo 08:57:12 or same repo with different tag 08:57:15 as you wish 08:57:27 opnfv/functest_arm:latest 08:57:40 jose_lausuch: I think https://jira.opnfv.org/browse/FUNCTEST-621 can be closed. 08:57:44 #info use new repo for arm functest builds opnfv/functest_arm:latest 08:57:45 and yardstick will do opnfv/yardstick:danube 08:58:19 ollivier: done 08:58:23 and we may reconsider this in E but I think it will be more clear to have dedicated repo >test>_arm 08:58:39 2 minutes left 08:58:41 yes 08:58:43 aob? 08:58:47 Status on feature projects? 08:58:50 you added this? 08:58:50 ok 08:58:52 #topic Status on feature projects 08:58:56 yep 08:59:05 quickly 08:59:11 ollivier, juhak, SerenaFeng: is it ok to be notified if the image build fails? 08:59:19 Do we have a good view on the feature project we may have to deal with for Danube 08:59:40 morgan_orange: there are not many more than Colorado.. 08:59:54 same features basically, with improvements/new tests 08:59:57 if so it could make sense to create a Jira and distribute them among us to have a counterpart to a feature project 09:00:13 + the mano related projects 09:00:23 ya 09:00:40 some projects may even not be there for Danube.1.0 09:00:40 https://jira.opnfv.org/browse/FUNCTEST-353 09:00:42 e.g. moon 09:00:51 HelenYao, sure 09:00:59 I will +2 to it 09:01:04 SerenaFeng: thx 09:01:08 for promise for instance, still not clear 09:01:18 ollivier, juhak, SerenaFeng: is it ok to be notified if the image build fails? 09:01:32 fine for me 09:01:43 juhak: great 09:01:58 https://gerrit.opnfv.org/gerrit/#/c/27271/ 09:02:06 #info most of the feature projects for Danube are known (only additional tests + mano related projects) 09:02:25 promise not clear? 09:03:08 I assume for the moment we keep the same tests, but gerald mentioned they were refactoring 09:03:17 ya right 09:03:25 I was just wondering when we could really test the target 09:03:34 MS5 is this friday 09:03:40 shall we spend time to refactor (according to our abstraction) or wait for the refactoring 09:03:41