08:00:02 #startmeeting Functest weekly meeting June 27th 2017 08:00:02 Meeting started Tue Jun 27 08:00:02 2017 UTC. The chair is jose_lausuch. Information about MeetBot at http://wiki.debian.org/MeetBot. 08:00:02 Useful Commands: #action #agreed #help #info #idea #link #topic. 08:00:02 The meeting name has been set to 'functest_weekly_meeting_june_27th_2017' 08:00:05 morgan_orange: is this the fix for the dashboards issue we talked about yesterday? https://gerrit.opnfv.org/gerrit/#/c/36527/ 08:00:06 #topic role call 08:00:26 #info Juha Kosonen 08:00:27 jlinkes: that fixes the iterations problem 08:00:30 #info Jose Lausuch 08:00:35 #info Cristina Pauna (Enea) 08:00:44 jlinkes: yes 08:01:06 morgan_orange: could you re-generate http://testresults.opnfv.org/reporting2/display/danube/functest/status-apex.html then, please? 08:01:08 #info Linda Wang 08:01:18 #chair morgan_orange 08:01:18 Current chairs: jose_lausuch morgan_orange 08:01:18 jlinkes: basicaly pagination has been introduced in the API then when I was colelcting the results, I always got the first page by default that is why the number of iterations and scenarios was no more correct 08:01:20 #info SerenaFeng 08:01:42 jlinkes: I tested the fix on http://testresults.opnfv.org/reporting2/display/danube/functest/status-apex.html, seems OK 08:01:45 #info agenda for today: https://wiki.opnfv.org/display/functest/Functest+5.+Meeting#Functest5.Meeting-27/06(8UTC) 08:01:52 #info Morgan Richomme 08:02:17 #topic Action Point follow-up 08:02:24 #info AP SerenaFeng jose_lausuch align in TestAPI authentication, next steps. 08:02:33 #info We brought the topic to the Test working group and Trevor will help us identifying the requirements. 08:02:54 Trevor said he will create a wiki page for it, 08:02:57 morgan_orange: sorry, I meant version1: http://testresults.opnfv.org/reporting/functest/release/danube/index-status-apex.html 08:03:03 do you know what's the progress? 08:03:13 #info Trevor will create a wiki with proposal/questions 08:03:13 jlinkes: in progress... 08:03:21 I don't know, I'll try to talk to him today 08:03:24 morgan_orange: thanks 08:03:28 ok 08:03:30 #info Jun Li 08:04:05 #topic Framework status 08:04:17 let me post some information 08:04:24 and we can then comment on the different topics/issues 08:04:29 #info major changes in the framework have been merged and some bugs have been fixed 08:04:38 #info Floating ips issue: newer nova client doesn't support floating ip creation. It's now managed by neutron client --> Fixed in Functest utils and SNAPS 08:04:50 #info Glance images issue: newer nova client doesn't support image creation. It is now managed by glance client --> Fixed in Functest utils and SNAPS 08:04:52 #link https://wiki.opnfv.org/display/dovetail/Dovetail+Test+Result+Database created by trevor 08:05:08 MatthewLi: ah, ok! Thanks! 08:05:13 :) 08:05:33 Great thanks MatthewLi 08:05:59 #info Public endpoint stauts (no admin access). Now supported in Functest but we still need to enable it in SNAPS: https://gerrit.opnfv.org/gerrit/#/c/36441/ 08:06:02 my favor 08:06:10 LindaWang: any comments about it? 08:06:26 jose_lausuch: I think i have fixed it 08:06:39 by reseting interface to public by default 08:06:47 aha ok 08:07:07 we still need confirmation from Steven 08:07:12 does anyone has an apex deployment? 08:07:15 Sure 08:07:33 jose_lausuch: I have already got one 08:07:42 I mean, to help testing it 08:07:42 both v2 and v3 are supported. 08:07:46 ok 08:07:52 ok, that sounds good 08:08:06 I think I can borrow a deployment from a colleague for some hours 08:08:09 I can also test that patch 08:08:16 Thank you 08:08:27 #info HTTPS: supported in Functest and now enabled in SNAPS: https://gerrit.opnfv.org/gerrit/#/c/36221/ 08:08:34 thanks LindaWang for this work 08:09:29 any comments ? 08:10:32 thanks LindaWang 08:11:01 Steven helps a lot 08:11:54 #info added unit tests for healtcheck tier. Please help review: https://gerrit.opnfv.org/gerrit/#/c/35661 there are 2 +2's . I plan to merge today 08:12:35 ok, let's talk about the Refstack status 08:12:38 #topic Tempest & Refstack in Functest 08:12:45 let me give some background information for everyone 08:12:51 #link https://gerrit.opnfv.org/gerrit/#/c/36463/ 08:12:56 #info We need to have the tempest repository in the container for Rally to install it and create the Rally verify environment. 08:13:23 #info This has been discussed with Rally team, there is no workaround. If we don't clone it, Rally will pull the repository from the internet anyway when configuring rally verify (which is against going Functest offline). 08:13:30 #info Now we clone Tempest in the Dockerfile 08:13:53 #info When we install Refstack, it also clones the repostory and it creates the .venv directory (that is the main difference between this directory and the tempest repo we clone in the Dockerfile) 08:14:00 #info The path to Tempest used by Refstack is in /home/opnfv/.tempest 08:14:14 #info The path to where we clone Tempest is /home/opnfv/repos/tempest 08:14:41 #info Refstack script hardcodes that path. It is not configuratble. See https://github.com/openstack/refstack-client/blob/master/setup_env#L57 08:14:47 now, comments :) 08:15:59 is it possible to convince refstack to make it configurable ? 08:16:44 We should talk with refstack team to check that. 08:16:54 or else it seems like we have no choice but to leverage link 08:16:56 I can do that 08:17:00 I know the PTL 08:17:04 I think it should be ok 08:17:11 I think so 08:17:14 but it's not there for now, so we need to come up with something in the meantime 08:17:24 But the symlink is also a good option. 08:17:38 what do you mean? 08:17:45 so, we remove the clone of tempest 08:18:00 and create a symlink from /home/opnfv/repos/tempest to /home/opnfv/.tempest ? 08:18:08 currently, symlink is ok, as we did in that patch 08:18:33 the only difference between those 2 directories is the .venv that refstack installs 08:19:13 Do you mean you want to create verifier by /home/opnfv/.tempest 08:19:14 ? 08:19:34 would it work? 08:19:58 rally verify create-verifier --source /home/opnfv/repos/tempest --name opnfv-tempest --type tempest --system-wide 08:20:08 rally verify create-verifier --source /home/opnfv/.tempest--name opnfv-tempest --type tempest --system-wide??? 08:20:10 I am not sure 08:20:18 Maybe we should check. 08:20:30 ln -s /home/opnfv/.tempest /home/opnfv/repos/tempest 08:20:52 so 08:20:53 2 options 08:21:07 1) we change the rally verify command pointing to /home/opnfv/.tempest 08:21:08 or 08:21:22 I think tempest is also installable, am I right? 08:21:25 2) we remove tempest repo in the image and create symlink ln -s /home/opnfv/.tempest /home/opnfv/repos/tempest 08:22:01 in both cases, we remove the tempest repo in the image 08:22:07 I think so 08:22:07 I have checked, rally verify create-verifier --source /home/opnfv/.tempest--name opnfv-tempest --type tempest --system-wide 08:22:11 It is ok 08:22:22 does it work? 08:22:25 yES 08:22:36 what sall we do then? 08:22:40 option 1? 08:22:47 and remove /home/opnfv/repos/tempest ? 08:22:48 I prefer the first 1 08:23:01 ok 08:23:04 +1, also waiting for your tests. 08:23:14 yes 08:23:20 I can create a patch with that change 08:23:27 when is /home/opnfv.tempest create? 08:23:31 by refstack? 08:23:40 After refstack is installed. 08:24:08 SerenaFeng: https://gerrit.opnfv.org/gerrit/gitweb?p=functest.git;a=blob;f=docker/Dockerfile;h=4c03a697d84be6e7a491e47fd6320970032ade2c;hb=refs/heads/master#l139 08:24:13 so it means that the installation order will be important and we create a dependency Refstack/Rally, right? 08:24:13 it is created here 08:24:17 by the setup_env script 08:24:30 morgan_orange: not really 08:24:44 Rally needs the repo when doing functest env prepare, not in the docker build 08:25:09 rally verify create-verifier will be executed in "functest env prepare" 08:25:16 ok 08:25:56 but yes, the dependency is that if some day we remove refstack from Functest (which I dont think it will happen), rally will fail 08:26:02 but we can always add the repo back 08:26:11 but that is unlikely to happen 08:26:23 so, I'm fine with relying on option 1 08:26:35 Aha 08:26:47 Sounds more reasonal 08:27:16 typo: reasonable 08:27:59 I would recommend to install tempest if possible, then make symlink for both rally and refstack if needed 08:28:25 but the way Rally manages it is different 08:28:26 Then you will make it more complicated. 08:28:27 in that case the tempest version we leverage will not rely on either rally or refstack 08:28:34 Rally only needs the tempes repo, not the installation 08:28:47 we can control the version anyway 08:29:08 the point is decouple rally/refstack 08:29:09 When it comes to refstack, the tempest version is supposed managed by restack. 08:29:16 let them rely on tempest directly 08:29:28 I don't think it will make it more complicated 08:29:53 That is why there is a ARG REFSTACK_TAG in Dockerfile. 08:30:11 Actually REFSTACK_TAG refers to tempest version 08:30:51 But we hope it should be controlled by refstack 08:30:58 why? 08:31:13 why it should be controlled by refstack? 08:31:40 refstack and rally should execute the same tempest version in my opinion 08:31:46 refstack is only a tool to drive tempest, it should be able to drive any tempest version 08:31:57 jose_lausuch totally agree 08:32:05 and it can not be determined by refstack 08:32:55 jose_lausuch: we can not make sure higher version of tempest will be run successfully by refstack sometimes 08:33:13 I am looking at the setup_env 08:33:13 https://github.com/openstack/refstack-client/blob/master/setup_env#L104 08:33:18 it seems it clones latest 08:33:28 git clone https://github.com/openstack/tempest.git ${TEMPEST_DIR} 08:33:28 We want to make it more controllable. 08:33:36 it depends on us to choose the proper version, but it doesn't mean to be controlled by refstack 08:34:04 then, option 1) is not valid 08:34:09 jose_lausuch: tempest repo here should be 15.0.0 08:34:21 But you git cloned the latest version 08:35:05 I am confused: git clone --depth 1 -b $TEMPEST_TAG https://github.com/openstack/tempest.git ${REPOS_DIR}/tempest 08:35:18 in the setup_env script from refstack, it is possible to specify the tempest installation 08:35:28 maybe SerenaFeng is right if we install tempest ourselves 08:35:37 python virtualenv.py ${TEMPEST_DIR}/.venv --python="${PYPATH}" 08:35:56 I just check the setup_env 08:36:10 it seems like what we discussed here is not supported 08:36:24 they hardcode the directory to .tempest 08:36:29 because in refstack it checks if tempest_dir exist 08:36:34 it will remove it first 08:36:48 then git clone a new latest one 08:36:52 right 08:37:06 after seen this doscussion, actually not get the main point. refstack use tempest, do many env prepare actions by https://github.com/openstack/refstack-client/blob/master/setup_env, and it may changes as master code grows. let's inherit refstack steps directly without trying to add or amend the steps by using own defined steps 08:37:22 NO 08:37:29 Not the latest 08:37:30 git checkout $CHECKOUT_POINT || if [ $? -ne 0 ]; then exit 1; fi 08:37:58 ah ok, great 08:38:06 we can give the commit ID to Refstack script 08:38:10 CHECKOUT_POINT="-q ${OPTARG}, Here OPTARG can be changed by -t parameter 08:38:11 that makes things easier 08:38:27 MatthewLi: I think we should use a fixed tempest version, not latest 08:38:43 jose_lausuch: Why? 08:38:46 jose_lausuch: yep we use 15.0.0 not latest 08:38:50 It is 15.0.0 08:38:52 YES 08:39:06 so 08:39:11 setup_env -t ${REFSTACK_TAG} 08:39:19 1) we change the rally verify command pointing to /home/opnfv/.tempest 08:39:20 REFSTACK_TAG=15.0.0 08:39:40 that's also defcore 2016.08 requirements 15.0.0 08:39:44 #info agreed to change the rally verify command pointing to /home/opnfv/.tempest and removing the clone of tempest repo 08:39:48 is it right? 08:40:04 YES 08:40:23 #info this way we use 15.0.0 version in Rally-tempest and Refstack 08:41:43 ok? 08:41:48 everyone agrees? 08:41:51 currently, ok 08:41:52 ok 08:41:53 AGREE 08:41:56 we have no choice 08:41:57 +1 08:41:59 as for the rally, if decide to do so, not object now, however, don't know if have problems. at least separate tempest repos for refstack and rally will not have problems :) 08:42:48 MatthewLi: why separating repos? this way we don't duplicate files and we control the version only 1 time 08:43:26 better to leverage only 1 tempest 08:43:31 Now there is only one tempest repo. 08:43:52 ok 08:44:02 who will work on that change? 08:44:05 We'd better be careful about tempest verison in future. It affects a lot. 08:44:05 I volunteer 08:45:25 #topic Functest offline 08:45:37 #info help review this https://gerrit.opnfv.org/gerrit/#/c/35593/ 08:45:44 what is the status in offline LindaWang ? 08:45:47 what is missing? 08:46:10 Only healthcheck tier 08:46:28 so , that patch 08:46:32 There are also 2 large imags(ubuntu and centos) needed 08:46:50 https://gerrit.opnfv.org/gerrit/#/c/35593/ 08:47:06 They are needed by 3 tests from snaps_smoke 08:47:31 We need to discuss with Steven. 08:47:41 ok 08:47:53 I wouldn't like to abandon that patch 08:47:57 he is tired of rebasing 08:48:01 we are not providing enough feedback 08:48:03 Whether remove the 3 tests or download these images by download_images.sh 08:48:09 Ryota Mibu proposed functest: enable doctor-notification test case https://gerrit.opnfv.org/gerrit/36561 08:48:24 I would put those images in dowload images.sh 08:48:30 I made some comments there. 08:48:31 no need to remove the tests 08:48:54 ok 08:49:48 jlinkes: it is done http://testresults.opnfv.org/reporting/functest/release/danube/index-status-apex.html 08:50:01 LindaWang: you mean , which tests need to be removed? from snaps_smoke ? 08:50:28 morgan_orange: thanks 08:51:03 test_dual_ports_dhcp (snaps.openstack.tests.create_instance_tests.CreateInstancePubPrivNetTests) 08:51:03 test_apply_simple_playbook (snaps.provisioning.tests.ansible_utils_tests.AnsibleProvisioningTests) 08:51:03 test_apply_template_playbook (snaps.provisioning.tests.ansible_utils_tests.AnsibleProvisioningTests) 08:51:10 please all read latest comments here https://gerrit.opnfv.org/gerrit/#/c/35593/ and provide feedback 08:51:13 Yes, from snaps_smoke 08:51:18 I think we discussed before that large images will be processed with volume, no ? 08:51:21 question is: shall we keep them or remove them? 08:51:30 yes 08:51:33 that is the idea 08:52:30 ok, let's discuss in the gerrit commit 08:52:32 then what's the problem now? 08:52:55 #info move large images to download_images.sh 08:52:57 #AoB 08:53:00 no problem 08:53:13 #info morgan_orange fixed iterations in the Functest Dashboard. 08:53:14 thank you! 08:53:25 #info it seems that we cannot generate anymore Danube docker file... 08:53:33 will we discuss the rest api? 08:53:59 is the info on wiki agreed by all of us? 08:54:09 yes for me 08:54:15 HelenYao: we can, I didn't see further comments on it, so I'm fine with it 08:54:27 if someone doesn't agree, shout 08:54:31 :) 08:54:52 another thing is, i am leaving this week. I would like to hand the work over to Linda 08:55:11 I will shout at him loudly 08:55:34 HelenYao: but will you still do things in Functest? 08:56:00 yes. when is the rest api supposed to be ready? 08:56:17 there is no date, everything is asap :D 08:56:37 I will still in Functest 08:56:42 how is the progress of testapi? 08:57:11 MatthewLi: this is about a new feature, the rest api for functest, not the Test api 08:57:15 Linda will be more focused than me. I would be glad to work with her on rest api 08:57:24 ok 08:57:27 #by now, can satisfy the requirement in https://wiki.opnfv.org/display/dovetail/Dovetail+Test+Result+Database 08:57:33 how long will you be off? 08:57:44 next week 08:57:57 the whole week 08:58:00 ok 08:58:07 jose_lausuch, SerenaFeng: yep I mean the testapi not restapi 08:58:26 what requirement? 08:58:29 cannot satisfy now 08:58:34 I just know it :) 08:58:34 authentication? n 08:58:35 no 08:58:59 SerenaFeng: so when will finish that? need which resource? 08:59:14 I will finish it asap 08:59:22 and I still need requirements from landing page 08:59:23 it's not an easy task 08:59:39 it seems there is no need for landing page in this wiki 08:59:49 maybe we need that in urgent, for landing page do u need which resource? 09:00:03 I don't know 09:00:28 I will discuss with Morgan and Jack 09:00:35 let's focus on authentication for testapi, and leave landing page for later 09:00:39 little confused.... OK, thanks.... 09:00:50 morgan_organge any requirements from landing page 09:00:55 I really CVP should provide people to work on it, not us doing all the work 09:01:10 jose_lausuch: yep let's let's focus on authentication for testapi, and leave landing page for later 09:01:13 a landing page for CVP should be managed by CVP, not by Functest 09:01:21 SerenaFeng: landing page Dovetail or landing page testing working group? 09:01:38 the one Jack works on 09:01:50 but that's a different thing 09:01:50 ok so the one from the testing group 09:01:54 I remember you mentioned that you want to make some update on scenarios 09:01:54 that is the test catalog, right? 09:02:10 catalog, test list customization, ... 09:02:13 I take the point 09:02:15 #info focus on authentication for testapi, and leave landing page for later 09:02:16 yes, that one 09:02:17 https://wiki.opnfv.org/display/dovetail/Dovetail+Test+Result+Database 09:02:18 I send a mail 09:02:21 I will contact Jack to fully support this work 09:02:46 ok 09:02:47 Hi, the requirements from landing page is also here. 09:02:48 time's up 09:02:48 JackChan_ also included in the same wiki page? 09:02:54 ok 09:02:56 SerenaFeng: yes 09:02:57 I will check that 09:03:18 jose_lausuch: how can we do to generate a Danube functest docker? 09:03:55 morgan_orange: https://build.opnfv.org/ci/view/functest/job/functest-docker-build-push-danube/ 09:04:14 it is failing 09:04:15 yep but error with bgpvpn 09:04:19 I will check 09:04:30 #action jose_lausuch check problem with docker image build in danube https://build.opnfv.org/ci/view/functest/job/functest-docker-build-push-danube/ 09:04:34 2 attempts (orchestra change + exclude Domino for Apex) 09:04:45 ok 09:04:48 I'll work on that 09:04:49 thanks all 09:04:50 @serena i check the openstackid, and it supports OAuth2.0, that can be used for auth 09:04:50 leo_wang: Error: "serena" is not a valid command. 09:04:53 I think the problem is casuse by pylint in requirements.txt in bgpvpn repo 09:04:55 #action morgan_orange sync with JackChan_ for next steps on landing page 09:05:13 upgrade setuptools will work 09:05:16 SerenaFeng: there was not such things in Danube.. 09:05:25 SerenaFeng: i check the openstackid, and it supports OAuth2.0, that can be used for auth 09:05:26 okey 09:05:43 https://gerrit.opnfv.org/gerrit/gitweb?p=sdnvpn.git;a=blob;f=setup.py;h=a6372603e2050eef972b726e466c6bdd0b5a86de;hb=refs/heads/stable/danube 09:05:47 we can discuss offline 09:05:52 #endmeeting