08:00:10 #startmeeting Functest weekly meeting February 14th 2017 08:00:10 Meeting started Tue Feb 14 08:00:10 2017 UTC. The chair is jose_lausuch. Information about MeetBot at http://wiki.debian.org/MeetBot. 08:00:10 Useful Commands: #action #agreed #help #info #idea #link #topic. 08:00:10 The meeting name has been set to 'functest_weekly_meeting_february_14th_2017' 08:00:14 #topic role call 08:00:17 #info Jose Lausuch 08:00:19 Hello 08:00:23 #info Morgan Richomme 08:00:25 hello 08:00:33 #chair morgan_orange 08:00:33 Current chairs: jose_lausuch morgan_orange 08:00:37 #info Juha Kosonen 08:00:58 #info Helen Yao 08:01:00 #info Cristina Pauna (Enea) 08:01:21 #info Linda Wang 08:02:18 Morgan Richomme proposed functest: [WIP] Integration of orchestra_ims test https://gerrit.opnfv.org/gerrit/28587 08:03:19 #topic Action point follow-up 08:03:24 #info AP1 morgan_orange: suggest a cross project review of the documentation to the testing group. DONE 08:03:41 #info Cédric Ollivier 08:03:43 #info AP2 jose_lausuch check for security scan logs in apex 08:03:50 #info Still too long and too much output. See this: https://build.opnfv.org/ci/view/functest/job/functest-apex-apex-daily-master-daily-master/398/consoleFull 08:04:15 #action jose_lausuch contact Luke to see if we can reduce output 08:04:16 #info AP3 All: all (specially Luke ): review for sec scan logs https://gerrit.opnfv.org/gerrit/#/c/28019/ 08:04:23 #info Merged 08:04:36 sec scan is not longer hosted in functest 08:04:44 code removed 08:04:48 Bonjour 08:04:49 #info AP jose_lausuch create a wikipage for logging best practicies for Functest. Not done. 08:04:54 good morning 08:04:58 #action jose_lausuch create a wikipage for logging best practicies for Functest 08:05:07 #info morgan_orange moon refactoring (Jira + FeatureBase) 08:05:12 status? 08:05:26 we should add a wiki page about review best practices as well 08:05:47 jose_lausuch: done 08:05:48 ollivier: you want to write it? 08:05:52 #info Done 08:05:54 jose_lausuch: ok 08:05:56 #info done 08:05:58 #link https://gerrit.opnfv.org/gerrit/#/c/28219/2 08:06:08 #info AP6 morgan_orange check moon status. Might need test refactor if it's in Danube. 08:06:17 #info not refactored but excluded from Danube release 08:06:26 #info refactoring could be done later with the new version of moon 08:06:37 #info AP7 SerenaFeng remove multisite.py. DONE: https://gerrit.opnfv.org/gerrit/#/c/28353/ 08:06:40 ok 08:06:40 thanks 08:06:48 ollivier: can I action you? 08:07:06 #action ollivier add a wiki page about review best practices 08:07:16 #topic Troubleshooting status (short update) 08:07:30 #info status apex http://testresults.opnfv.org/reporting/functest/release/master/index-status-apex.html 08:08:01 #info Functest apex has been executed last days (last time 10 Feb) 08:08:17 #info bgpvpn scenario integrated but not working yet 08:08:21 #info work in progress 08:08:46 #info since it is not executed, we don't have results in the dashboard 08:08:47 shall we contact doctor team 08:09:03 it seems that we have issue with all the installers at the moment 08:09:08 yes 08:09:27 and parser has problems running in fuel 08:09:27 not all, there are PASS on fuel depending on the scenario 08:09:31 yes 08:09:35 nosdn kvm works 08:10:07 it works in some scenarios in fuel 08:10:08 oops problem with hyperling with fuel to get jenkisn details 08:10:27 we should add a topic about https://gerrit.opnfv.org/gerrit/#/c/28563/. Which is the installer which tiggered it? 08:10:29 can I action you on that? 08:10:44 ollivier: let's discuss it in the AoB topic 08:10:51 ollivier: but fuel and compass 08:10:57 maybe joid, I didnt look 08:11:01 apex didnt run last 3 days 08:11:38 #action morgan_orange fix hyperlink to jenkins on the dashboard 08:11:45 ok 08:12:08 #info lot of scenarios executed in fuel 08:12:09 a r is missing maste instead of master but probelm only on fuel...strange 08:12:15 morgan_orange: is stable/colorado frozen? Jenkins doesn't vote on the last patches 08:12:17 morgan_orange: shall we exclude noha ? 08:12:34 morgan_orange: https://gerrit.opnfv.org/gerrit/#/q/status:open+project:functest+branch:stable/colorado+topic:backport_odl 08:12:39 ollivier: colorado is not longer maintained by anyone 08:13:37 jose_lausuch: OK. But they must be merged.. at least to clear the list 08:13:46 ollivier: I will merge them 08:14:00 ollivier: but those patches will never be in the container, we already have frozen our release images 08:14:03 tehre was a discussion on keeping at least 1 gate for Colorado 08:14:20 but CI people said noone was working on colorado... 08:14:37 jose_lausuch: yes but it can be patched 08:14:44 morgan_orange: a patch has been merged several days ago 08:14:52 what patch? 08:15:25 https://gerrit.opnfv.org/gerrit/#/c/28163/ 08:15:27 do you want to update the colorado_3 image?? 08:15:32 I'm not sure about that 08:16:14 no. Just merge the patches. They can be downloaded via gerrit if required 08:16:24 merge done 08:16:44 ollivier: I prefer to spend time fixing danube, just that :) 08:17:22 morgan_orange: thank. I didn't think you could vote for jenkins :) 08:17:38 morgan_orange: please merge https://gerrit.opnfv.org/gerrit/#/c/23069/ too 08:17:45 #info status compass http://testresults.opnfv.org/reporting/functest/release/master/index-status-compass.html 08:17:54 HelenYao are you there? 08:18:00 yes 08:18:07 #info currently 2 scenarios in functest 08:18:15 do you have an update from compass? 08:18:50 I think odl_l2 has been activated lately 08:18:52 right? 08:18:56 compass is trying to run on centos and the snaps related healthcheck failed 08:18:58 old_l3 sorry 08:19:29 compass team is still working on odl_I3 08:19:41 #info compass is trying to run on centos and the snaps related healthcheck failed 08:19:45 #info compass team is still working on odl_I3 08:19:48 ok 08:19:59 not bad results in the last run https://build.opnfv.org/ci/view/functest/job/functest-compass-baremetal-daily-master/815/consoleFull 08:20:06 failures in tempest and rally 08:20:28 actually rally didnt work well at all 08:20:36 it is caused by heat 08:20:48 compass has delivered the code and it will be executed tomorrow 08:21:08 and tempest didnt execute at all 08:21:09 ? 08:21:32 2017-02-14 03:23:17,152 - Tempest - ERROR - Error with run: 'NoneType' object is not iterable 08:22:24 let me see 08:22:37 it's happening everywhere 08:22:38 I noticed that too: https://jira.opnfv.org/browse/FUNCTEST-725 08:22:56 juhak: can you look into that 08:22:56 jose_lausuch: agree to spend time fixing Danube. They were an old review which was stucked... Now odl dashboard is fine :) 08:23:07 there were a couple of patches about tempest 08:23:09 jose_lausuch: yes 08:23:25 #action work on tempest stopper https://jira.opnfv.org/browse/FUNCTEST-725 08:23:36 I think the error is caused by our logic of tempest 08:24:26 ok 08:24:34 #info status fuel http://testresults.opnfv.org/reporting/functest/release/master/index-status-fuel.html 08:25:16 morgan_orange: should we update the dashboard? 08:25:24 to show api_check and connection_check test cases? 08:25:34 it is already done 08:25:41 I need to add healthcheck 08:25:49 the tier? 08:25:58 oops 08:26:18 OK no since we moved them to healthcheck theya re not displacyed anymore 08:26:33 ya 08:26:38 I can try to reenable them 08:26:39 and we have now the snaps healtcheck as well 08:26:41 with dhcp test 08:26:46 shall we remove healthcheck.sh fromù the CI run? 08:26:47 ok 08:26:52 yes 08:26:58 but snaps healtcheck has still some problems 08:27:01 wait to fix them first 08:27:10 ok 08:27:14 but it would be nice to update the dashboard already 08:27:19 do you have time? 08:27:24 to many APs on you :) 08:27:24 #action morgan_orange add api_check and conenction_check in reporting 08:27:40 and you are taking days off.. 08:27:48 yes 08:28:01 #info SNAPS working on basic scenario in fuel 08:28:17 shoudl we also remove promise from the dashboard? 08:28:22 or we can wait 08:28:26 there is still time 08:28:37 #info status joid http://testresults.opnfv.org/reporting/functest/release/master/index-status-joid.html 08:28:51 #info: last joid successful RUN yesterday (lxd-noha): https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-master 08:28:57 #action morgan_orange see why huawei-pod12 is stil not seen as a CI POD 08:29:01 #info 3 scenarios 08:29:05 #info on this scenario only basic tests are OK 08:29:27 #info nosdn-nofeature-ha scenario FAIL during installation due to '' => should be fixed by your nightly patch 08:29:27 ok 08:29:37 I did not follow 08:29:55 #topic Refactoring status: ONOs/Moon 08:29:56 Unable to establish connection to http://10.5.14.23:5000/v2.0/tokens: ('Connection aborted.', BadStatusLine("''",)) 08:30:05 aha 08:30:08 ok 08:30:16 at least nosdn-nofeature doesnt look good 08:30:19 sorry 08:30:21 doesnt look bad 08:30:26 tempest 100% 08:30:35 in noha 08:30:57 when it runs 08:31:02 https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-master/926/console 08:31:19 ya 08:31:25 ok 08:31:32 what about refactoring of ONOS 08:31:38 that is a run done after your patch was merged 08:31:47 No route to host 08:32:01 we should discuss about that patch... 08:32:20 ollivier: let's follow the agenda, just 1 topic and we talk about it 08:32:33 for ONOS, we should find a contact in ONOS team, no? 08:32:46 HelenYao: how is your time? 08:32:49 not sure it is straight forward as for some feature projects 08:33:04 HelenYao: would you have time to look into ONOs test case or contact someone from ONOs team? to refactor the test case ? 08:33:27 jose_lausuch: I can help with that. 08:33:44 jose_lausuch: do you know who shall I reach out to in ONOS team? 08:33:55 @cores: please have a look https://gerrit.opnfv.org/gerrit/#/c/27463/ too. Another +2 is required 08:33:55 ollivier: Error: "cores:" is not a valid command. 08:34:26 https://gerrit.opnfv.org/gerrit/#/q/project:functest+onos 08:34:56 HelenYao: Wenbin Wu 08:35:06 he has written most of the code for onos 08:35:10 Merged functest: Refactoring of args and parser variable https://gerrit.opnfv.org/gerrit/27463 08:35:20 Qinglong Lan 08:35:22 jose_lausuch: sure. I will talk to him 08:35:32 HelenYao_: Qinglong Lan and Wenbin Wu 08:35:45 jose_lausuch: ok 08:35:54 HelenYao_: it would be nice to have it this week, but if they can't make it... it's ok 08:36:00 just to get rid of exec_tests.sh 08:36:04 which is stinky already :) 08:36:12 morgan_orange: what about Moon? 08:36:23 jose_lausuch: sue. I will try to wrap it up asap 08:36:23 is it in danube in the end or not? 08:36:23 it is excluded from the exec_tests 08:36:27 HelenYao_: thanks 08:36:37 it is not in Danube 08:36:41 #info Moon excluded already from exec_tests and not in Danube 08:36:41 so 08:36:43 patch was merged 08:36:56 only ONOS to refactor and we are 100% using TestCaseBase right? 08:37:40 jose_lausuch: https://gerrit.opnfv.org/gerrit/#/c/28219/ 08:37:50 morgan_orange: thank you. Cou you please review https://gerrit.opnfv.org/gerrit/#/c/27113/ too ;) ? 08:37:59 morgan_orange: thanks, i didn't see that 08:38:41 #info only ONOS and ONOS-SFC to be refactored and be 100% using the new framework 08:38:46 let's try to make it guys :) 08:38:54 ok 08:38:55 #topic Plugfest/Hackfest planning 08:38:56 \o/ 08:39:05 Merged functest: Add ODL netvirt connectivity suites to list of robot tests https://gerrit.opnfv.org/gerrit/27113 08:39:24 #info Plugfest in France: http://events.linuxfoundation.org/events/opnfv-plugfest 08:39:30 register if you can attend 08:39:40 we should prepare something 08:39:53 besides what we always do, setting up a DB to push results 08:40:02 I did not know there were montains in Paris... :) 08:40:04 we should also prepare a Plugfest-Dashboard 08:40:14 at the end of everyday, there is a recap of the day 08:40:23 and it would be nice to show the results on the screen for everyone 08:40:37 jose_lausuch: I think it would be the right place to finalize the landing page / Test case catalog and maybe introduce kumar's work on VNF catalogue? 08:40:45 yes 08:40:47 as well 08:41:06 #info planning topic: Finalize landing page 08:41:22 #info planning topic: present Kumar's work on VNF catalog 08:41:46 we planned to do it beginning of March 08:41:51 #info planning topic: presentation about new Functest framework and how easy is to integrate test cases 08:41:55 here or during the testing weekly meeting? 08:42:01 not here 08:42:03 morgan_orange: a last one? https://gerrit.opnfv.org/gerrit/#/c/28565/ 08:42:05 for the testgroup, not here 08:42:16 we will follow up 08:42:20 we need to further discuss 08:42:22 there is still time 08:42:28 but just an early reminder for everyone 08:42:41 register and think about topics to be presented, or things we can show/offer 08:43:18 any comment on the plugfest/hackfest? 08:43:32 ollivier: argh unix permission one more time... 08:43:57 please team, be aware of that when submitting a patch (including myself) 08:44:01 jose_lausuch: lots of good ideas, see offline how we could have a daily dashboard 08:44:20 Merged functest: Fix again unix permissions https://gerrit.opnfv.org/gerrit/28565 08:44:24 morgan_orange: sure. At least one dev is using windows... This is the first check I will put on the wikipage 08:45:01 ollivier: maybe next time dont say "again" in the commit message, it sounds you are pisseed (that you might be, but don't tell publicly) :) 08:45:04 hehe 08:45:19 jose_lausuch: No. It avoids copy paste a previous commit message 08:45:24 i felt very very for that 08:45:30 sorry 08:45:48 HelenYao_: don't worry. 08:46:08 #topic AoB 08:46:13 ollivier: shoot 08:46:50 http://pastebin.com/raw/zj8zJLJa 08:47:22 we can't say it doesnt concern functest since installers haven't changed anything in their RC file and suddenly nothing works 08:47:25 it does concern functest 08:47:38 No. Look at thge input file 08:48:04 It's wrong and we don't have to bypass their issues. 08:48:08 http://pastebin.com/raw/s9e4WVcR 08:48:28 "'key'" = "'value'" 08:48:36 ollivier: they have never changed that and it has always work, we can push such a big change at this point in time 08:49:07 *we can't 08:49:08 No before export and bash fix it... We don't have to fork a process for sourcing env vars :) 08:49:24 what do you mean? 08:50:37 The previous implementation was bad. As it relies on export in a new process, it automatically removes "" and '. 08:50:39 http://pastebin.com/raw/s9e4WVcR 08:50:44 sorry 08:50:59 Now we are correctly parsing the file so "'value'" is interpreted as 'value' 08:51:23 if password is "'foo", it must be interpreted as 'foo 08:51:27 apex: http://pastebin.com/raw/fSUT6qSk 08:51:42 yes 08:51:56 but I prefer fixing from what we have now than reverting to break again CI 08:51:56 In you case you remove all ' in keys and values 08:51:59 it also failed in Compass 08:52:04 fortunatelly 1 time only 08:52:07 2 or 3 in fuel 08:52:18 apex didnt run yesterday 08:52:24 same for joid 08:52:31 Yes. And? 08:52:47 if we want installers to produce a RC file without single quotes is too late 08:53:02 we can push it for E-release 08:53:12 I have never seen a bash file containing export "'key'" = "'value'" 08:53:26 it is actually 08:53:29 "\key" = "'value'" 08:53:34 "key" = "'value'" 08:53:37 but again 08:53:43 I thought it's "'jey'" 08:54:04 ollivier: http://pastebin.com/raw/s9e4WVcR 08:54:06 that is from fuel 08:54:10 it's always like this 08:54:26 I can agree it's not maybe the right thing to do 08:54:46 And? I can show thousands of bash files :) 08:54:51 but I dont agree we have to make our framework complain about it at this ppoint and blame the installer temas 08:55:27 others? opinions? 08:55:42 I am lost.. 08:55:53 So please revert my patch and fork again a new process 08:56:21 rc file has not a good format, but it was always the case 08:56:24 ollivier: your patch is fine, but we just need to read the values like before 08:56:34 then we introduce something to manage properly tht triggers issues 08:56:36 no your patch creates other issues 08:56:44 you remove all ' in values 08:56:48 then it breaks the CI 08:56:50 morgan_orange: this is a good formed rc file: http://pastebin.com/raw/fSUT6qSk This is bad: http://pastebin.com/raw/s9e4WVcR 08:57:22 ollivier: can you push a patch to remove only the ' chars at the beginnig and the end if they exist? 08:57:56 jose_lausuch: I do not see difference in the 2 files.. 08:57:57 it might introduce issues, but it's a bigger issue to break the CI IMO 08:58:03 morgan_orange: look at the quotes 08:58:09 Sure I can. But we have to discuss at an higher level that functest shouldn't bypass the installers limitations.... 08:58:16 OS_AUTH_URL='http://192.168.0.34:5000/v3' vs OS_AUTH_URL=http://192.168.37.17:5000/v2.0 08:58:51 ok the links should not be OK both are OS_AUTH_URL=http://192.168.37.17:5000/v2.0 08:58:54 but OK I see 08:58:58 Inputs must be clear and respect common standards 08:58:59 ollivier: yes you are right, but we don't have to cut them this way, we have to collaborate with them and that takes time 08:59:27 time that we dont have for this release 08:59:45 so I suggest to check for the values, if they have ' chars at the begginign and the end, we remove them 08:59:49 we must add a big warning log " Functest made uggly adaptations to make it work...thank your for the fishs" 09:00:32 but Ok I see your point jose_lausuch it is too late MS3 is passed since several weeks..; :) 09:00:33 I will add UT and put the minor modifications to limit impacts but we have to open a discussion at an higher level for the next release 09:00:40 morgan_orange: we didnt make uggly adaptations, we had the ugly adaptations from the beggining, the RC file has been always the same 09:00:43 maybe we should cretae a JIRA towards all the installer for E release 09:01:00 ollivier: and we will do, for the E-river planning for sure 09:01:25 we have to find these and more limitations and try to push them to the installer teams, and help fixing them of course 09:01:28 same story, I saw HelenYao comment on promise. I would like to merge it and let them do the adaptations 09:01:29 And we have to stop hacking functest when it fails for external reasons 09:02:32 ollivier: ya, but in this case we didn't "hack" it, the source credentials thingy has always work (maybe using ugly forks and so on..). 09:02:33 but ok 09:02:35 functest scope is to run the test and with teh new framework it is relatively easy to split the responsability area 09:03:11 I created the patch on promise in order to initiate the work and help but it should be under feature project responsability assuming that it can be run now 09:03:17 yes. Thank export and bash we remove the "' ... 09:04:06 I have to drop to another meeting 09:04:10 ok 09:04:10 see you in 8 days 09:04:12 thank you all 09:04:15 have a good vacation 09:04:17 #endmeeting