08:00:24 #startmeeting Functest weekly meeting September 13rd 08:00:24 Meeting started Tue Sep 13 08:00:24 2016 UTC. The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot. 08:00:24 Useful Commands: #action #agreed #help #info #idea #link #topic. 08:00:24 The meeting name has been set to 'functest_weekly_meeting_september_13rd' 08:00:28 #topic call role 08:00:33 #info morgan Richomme 08:00:36 #info Viktor Tikkanen 08:00:40 #info Juha Kosonen 08:00:45 #info Cristina Pauna (ENEA) 08:00:52 #info Jose Lausuch 08:00:58 #info Luke Hinds 08:01:19 #info agenda https://wiki.opnfv.org/display/functest/Functest+Meeting 08:01:29 is Rohit connected? 08:01:43 A comment has been added to a proposed change to functest: Push FAIL to DB for onos-sfc https://gerrit.opnfv.org/gerrit/20851 08:01:48 any topic you want to add? 08:01:59 we may notice the first meeting of our new friend OPNFV-Gerrit-Bot 08:02:09 #topic action point follow-up 08:02:09 :) 08:02:23 #info Jun Li 08:02:35 #link http://ircbot.wl.linuxfoundation.org/meetings/opnfv-functest/2016/opnfv-functest.2016-09-06-08.00.html 08:02:42 #info AP1 jose_lausuch ollivier find best way to coloradoize our docker and try to keep a master to be able to test changes without breaking colorado gate 08:02:47 #info done, using arg, 2 docker files availables 08:02:57 #info AP2 jose_lausuch sync for apex/bgpvpn 08:03:09 1sc 08:03:18 created a jira for that 08:03:54 #info AP3 jose_lausuch sync for apex/sfc 08:04:24 https://jira.opnfv.org/browse/APEX-219 08:05:20 #info apex-sfc has the same status as fuel-sfc, there has been some failures in the boron releases, currently testing Boron RC3.5 which came out yesterday and it's the release candidate 08:05:24 OK we should probably reference this JIRA in the release note 08:05:31 #info patch submitted for SFC, new try, new hope... 08:05:40 #info AP4 May-meimei sync for apex/onos and apex/onos-sfc 08:05:41 #info if that still gives us problems, we will report that in the release notes and leave the test failing (known upstream bug) 08:06:16 onos and onos-sfc not run anymore in apex/Colorado 08:07:17 morgan_orange: stop the jobs of sfc? 08:07:43 #info raghavendrachari 08:07:49 no jobs are still there but not passing healthcheck 08:07:49 A comment has been added to a proposed change to functest: [ODL-SFC] Add push resulst to DB https://gerrit.opnfv.org/gerrit/20929 08:08:00 https://build.opnfv.org/ci/view/apex/job/functest-apex-apex-daily-colorado-daily-colorado/157/console 08:08:40 I was wondering if somebody on onos side was looking at the issue 08:09:07 Jose Lausuch proposed functest: [ODL-SFC] Add push resulst to DB https://gerrit.opnfv.org/gerrit/20929 08:09:11 it seems to affect only apex 08:09:15 A comment has been added to a proposed change to functest: [ODL-SFC] Add push resulst to DB https://gerrit.opnfv.org/gerrit/20929 08:09:16 A comment has been added to a proposed change to functest: [ODL-SFC] Add push resulst to DB https://gerrit.opnfv.org/gerrit/20929 08:09:22 sorry for the spam :D 08:09:30 A comment has been added to a proposed change to functest: [ODL-SFC] Add push resulst to DB https://gerrit.opnfv.org/gerrit/20929 08:09:41 #action morgan_orange sync with apex/onos 08:09:47 #info AP5 May-meimei sync compass/onos-sfc 08:09:51 ok guys, let's stop working during the meeting 08:09:59 #info idem AP4 08:10:04 #info AP6 SerenaFeng sync fuel/doctor 08:10:04 A comment has been added to a proposed change to functest: [ODL-SFC] Add push resulst to DB https://gerrit.opnfv.org/gerrit/20929 08:10:16 there is a pending patch 08:10:23 yeah, 08:10:34 they are still disscussing about the process 08:10:36 today fuel/doctor prevents at least 6 scenarios to be run 08:10:53 #link https://gerrit.opnfv.org/gerrit/#/c/20663/ 08:10:56 I wanted to bring that up during the release meeting yesterday, but there was no time 08:11:35 #info discussions on how to fix doctor in fuel pending..is there any date before moving from Colorado 1.0 to colorado 2.0 08:11:44 cgoncalves: ping 08:11:53 #info https://gerrit.opnfv.org/gerrit/#/c/20627/ was abandoned 08:11:54 #info fuel team in the patch mentioned a possible fix beginning of this week from doctor 08:12:24 #info fuel scenarios are fine, it is just a decision from release management to see until when we try ... 08:12:44 #info AP done (it is more a question for release management, Doctor/fuel team) 08:12:51 #info AP7 viktor_t review scenarios where tempest is not fully sunny 08:13:16 #info AP8 juhak review scenarios where rally is not fully sunny 08:13:27 #info there were failed cases on os-nosdn-lxd-ha/noha scenarios, corrected along with FUNCTEST-460 and FUNCTEST-464 08:13:48 #info AP9 morgan_orange test rally black list on lxd scenario 08:13:53 #info done, lxd scenario now both OK for joid 08:13:57 #info AP10 morgan_orange add push to DB in odl_sfc 08:14:01 #info patch created by José, review in progress https://gerrit.opnfv.org/gerrit/#/c/20929 08:14:09 and currently under review.. 08:14:19 morgan_orange: we will need also some patching in the dashboard, right? 08:14:19 #action morgan_orange wait for merge before enabling it in reporting 08:14:24 adding the sfc test case to the scenario 08:14:28 or that's automatic/ 08:15:21 almost..https://git.opnfv.org/cgit/releng/tree/utils/test/reporting/functest/reportingConf.py 08:15:43 config adaptation to be planned 08:15:48 should not be long 08:15:55 #info AP11 SerenaFeng Raghav review Yardstick documentation 08:15:59 I saw only Serena review 08:16:00 ok 08:16:19 raghav: https://gerrit.opnfv.org/gerrit/#/c/20509/ 08:16:25 please review directly in gerrit 08:16:35 it patches a lot 08:16:36 yes am doign now .. today i'll finish 08:16:39 #action raghav review yardstick 08:16:49 #info AP12 morgan_orange add CristinaPauna and Alex Avadanii in Functest doc review 08:16:54 #info I didn't get any review invitation for documentation containg arm references 08:16:57 #info done for CristinaPauna, Alex not found in gerrit reviewers 08:17:29 #info https://gerrit.opnfv.org/gerrit/#/c/20315/ does not contain diffs for arm 08:17:59 #info I did see thoug in the existing docs some referenced to arm that I would like to modify 08:18:07 # should I do that directly? 08:18:18 Ok I have to check there are some in the doc but maybe not in the fiels under review 08:18:30 #action morgan_orange check arm ref and add them in the doc review for CristinaPauna 08:18:36 #info AP13 morgan_orange pushich topic 2 and 6 08:18:43 #info done +1 with topic on VNF catalog 08:19:12 #link https://wiki.opnfv.org/display/DEV/Intern-projects-page 08:19:18 #info AP14 everybody review and comment D wiki page 08:19:26 Focus on Colorado... to be discussed in next sections 08:19:38 any comments on the different action points? 08:19:46 nop 08:19:55 viktor_t: do you want to comment your AP? 08:20:18 I checked apex/os-odl_l2-sfc-noha case 08:20:37 probably will create a Jira case if Tim will not comment 08:20:50 ok 08:20:57 #topic Colorado 08:21:20 #info I reported to Release management that from Functest perspective, it would be possible to release today as most of the scenarios are OK and all the remaining errors are understood and documented in the release note. 08:21:55 maybe check onos on apex (could be considered as a regressiona s it used to work with brahmaputra as far as I remember and we also add successful runs in master...) 08:22:43 #info apex: first green scenario light on monday 08:23:17 #link http://testresults.opnfv.org/reporting/functest/release/colorado/index-status-apex.html 08:23:28 #info issues already discussed in AP follow up section 08:23:47 #info note all the odl_l3 scenarios will be failed whatever the installer 08:23:52 #info compass 08:23:55 #link http://testresults.opnfv.org/reporting/functest/release/colorado/index-status-compass.html 08:24:14 #info work in progress in moon (apparently Tempest OK and moon OK, issue on odl known and documented) 08:24:29 #info fuel 08:24:32 #link http://testresults.opnfv.org/reporting/functest/release/colorado/index-status-fuel.html 08:24:47 #info already some green scenarios...without doctor most of them will be green 08:25:10 #info fix on noha scenarios merged yesterday (test excluded in HA but not in noha) 08:25:16 #info joid 08:25:23 #link http://testresults.opnfv.org/reporting/functest/release/colorado/index-status-joid.html 08:25:29 #info most of the scenarios OK 08:25:47 note during the release meeting, tehre were some confusion between functest reporting and release reporting 08:26:01 people seem afraid of red... 08:26:17 yes, and this is only the functest view 08:26:27 whcih doesnt mean it has to be the release gate 08:26:36 I explained that green is only when everything is fine in Functest 08:26:53 even red status in Functest can be declared OK for a release (because we documented the issue) 08:27:06 but I do not see the point to put green just to put green.. 08:27:14 a gauge will be probably better 08:27:24 and less frightening 08:27:33 Danube improvement :) 08:27:55 but as mentioned in the introduction today, Functest will be OK for the release (last review to be done on user guide + release note) 08:28:16 are you OK with the statement (there are still open JIRA to be cleaned before the release..) 08:29:11 can we still delay our release? 08:29:12 meaning 08:29:27 can we say: we are ok, but if we release a bit later, we might have less bugs 08:29:37 or bugs to report in the release notes 08:29:57 it is always the case... :) 08:30:15 so yes we can say that but I would say that today there is no objection for releasing 08:30:35 ok 08:31:18 #info no objection for Colorado releasing from Functest 08:31:21 I have a request for the bgpvpn scenario 08:31:28 from the team 08:31:34 maybe for AoB 08:31:38 if there is time 08:31:53 ok 08:32:11 #topic Security Audit is coming... 08:32:27 #info in fact it came, errors shared with core developpers 08:32:52 #info no critical issues, yesterday patch abandonned due to confusion json.yaml load/safe_load 08:33:02 So we are almost there, last patch is on use of eval. But I believe we might be able to abandon this too.... 08:33:23 For anyone who wants to read the techincal backgound: http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html 08:33:33 Save me typing it all out. 08:33:46 I was happy...I expected hundreds of page from bandit ... 08:33:55 So the concern is, when eval is accepting input from an untrusted source 08:33:59 but at the end it seems that we did not bad... 08:34:02 For example, a webform. 08:34:12 But I believe this is not the case with the API code 08:34:29 it seems like its more being used for mongodb string filtering etc. 08:34:38 and ast.literal can not substitue all of eval works 08:34:49 So if we can confirm that no input comes externally, then we can abandon 08:35:08 I sustitue almost all of evals in testapi except mongodb access 08:35:09 I believe no input comes externally, but I am not the best person to know the code well. 08:35:24 ok 08:35:34 abandon means? 08:35:52 we do not have lots of interaction with external world except the test API 08:36:14 #topic D release: discussion with OAI 08:36:22 so we don't need to subsitute eval with literal_eval? 08:36:36 Rohit_openairint: mae a presentation on Open Air Interface last thursday during teh testing weekly meeting 08:36:41 Only if you take external input. 08:36:52 yes 08:37:04 from an untrusted source (the internet / a user outside) 08:37:05 #link https://wiki.opnfv.org/rest/documentConversion/latest/conversion/thumbnail/7767229/1 08:37:16 lhinds ok, I will abandon my patch 08:37:20 k, SerenaFeng will ping you after the meeting 08:37:28 Let's discuss there 08:37:37 lhinds ok 08:37:41 OAI developed several VNFs, I think we could consider the vEPC deployed through a JUJU VNFM for D release 08:38:28 my view is to follow vIMS best practices 08:38:36 we are currently looking at using FUEL to deploy OAI 08:38:53 yes, we are looking at vIMS scripts and see how to best integrate our code 08:38:58 do you have dependencies towards the installer? vIMS can be deployed on any installer 08:39:11 what I suggested is 08:39:14 For EPC, we will need 14.04 Ubuntu + 4.7.x kernel (for GTP module) 08:39:36 We need to use KVM only because of GTP kernel module 08:40:01 joid scenario supports xenial 08:40:04 In my opinion, if we deploy on FUEL, it can deployed on JUJU, or any other installer, correct? 08:40:09 yes 08:40:30 I would prefer 14.04 for the moment. We are working on supporting Xenial internally 08:40:45 ok 14.04 are also supported 08:41:05 We would also need minimum system requirements for deploying JOID on single machine 08:41:36 at least in my lab@Open 08:41:47 I am deploying OPNFV in single machine 08:42:12 ok you can also ask for access to community labs 08:42:45 but then you need to precise the resources you need and the scenario if needed (dependencies towards the controller?) 08:42:48 yes, but it will be also good to have our own local testbench for debugging, troubleshooting. At least for JOID, the requirements were very high 08:42:55 FUEL was okay on single machine 08:43:17 ok 08:43:29 do you see issues with using 4.7.x kernel compiled from source within OPNFV 08:43:34 for vEPC use case? 08:44:40 Is it also possible to deploy EPC on bare-metal compute nodes instead of KVM? 08:44:49 from OPNFV environment 08:45:07 if i is specific, you will probably need to create your own scenario 08:45:13 I see 08:45:21 like KVM or OVS 08:45:45 and this can be created from OPNFV functest scripts, right? 08:45:47 teh vIMS can be run on "generic" scenario, i.e. without kernel patching of the infrastructure 08:45:56 yes 08:46:04 no scenarios are more linked to the installers 08:46:17 functest is "just" deploying and running the tests 08:46:38 OK, I take a note of this 08:47:59 so to be clear, you do not need ressources (user/tenant/..) but a specific configuration of the infra? 08:48:13 the kernel change could not be only at the image level 08:48:15 I think we need to study OPNFV first 08:48:17 a bit 08:48:29 and then come back with some proposals 08:49:05 for specific kernel, we should use scenario which needs to be created via installer 08:49:09 concretely today you are not able to run your VNF in any commercial Cloud solution (as the kernel is not patched) or is the issue only for the image of your VNF 08:49:45 we can run it inside KVM (with modified kernel) 08:50:10 so looks like the kvm scenario to me 08:50:14 we can also have another approach only for functest. We have slightly older tag which uses only 3.19 generic kernel 08:50:28 we can use this as we do not need to do high throughput tests anyway 08:50:33 +1 on jose_lausuch AOB discussion on networking-bgp I am interested in that too 08:51:02 Ok, let's try to deploy it on an existing OPNFV and see the gap 08:51:09 OK 08:51:22 we do some analysis and come back with possible solutions to you 08:51:32 ok 08:52:04 #info discussion on vEPC/OAI integration - maybe a scenario is needed (kvm like) to be tested on existing OPNFV 08:52:13 #topic Meetup 08:52:29 last week we say that some of us will attend the OpenStack Summit 08:52:31 but not all 08:52:42 shall we plan a meetup as we did in Espoo 08:52:49 if so after or before openStack Summit 08:53:06 +1 for Laninon! 08:53:17 I'd say after OpenStack 08:53:19 discussions on D release started planning is supposed to be completed on the 13th of October... 08:53:26 wait 08:53:32 OS summit is at the end of OCtober... 08:53:40 mmm 08:53:40 scenario defined 3/11 08:53:49 test cases defined 18/11 08:53:54 oops 08:53:57 no 1/12 08:54:08 What about beginning of OCtobe? 08:54:12 installer integration completed for the 18/11 08:54:29 I did not react to the planning... 08:54:31 yet 08:54:39 still branching windows 08:54:54 Ok for a meetup in Lannion beginning of October 08:55:03 is it not a bit short / business trip? 08:55:23 why dont we create a pool? 08:55:29 and people put whats best 08:55:32 wht about the 6,7 or 13,14 if you want to stay over the week end... 08:55:34 ok 08:55:48 #action morgan_orange prepare pool for Functest meetup in Lannion 08:56:00 I think its the best option for now, since we can 08:56:04 can't decide it now 08:56:22 ok 08:56:25 #topic AOB 08:56:33 ho ahead with bgpvpn 08:56:40 ok, I'll explain our concern 08:56:41 so 08:56:49 https://jira.opnfv.org/browse/SDNVPN-54 08:57:04 #info most of the bgpvpn scenarios are failing in CI due to a timeout 08:57:26 #info the tests are working, its simply the calls to neutron/nova api take too long, but they work 08:57:59 #info for example, tempest 100% successful but takes 1 hour for the smoke, same for rally sanity 08:58:24 our idea was that we could disable rally testing for bgpvpn scenarios, and put a comment in the release notes 08:58:38 since rally is mostly dealing with performance of apis 08:58:42 not functionality 08:58:49 and we think that will be fixed in Boron 08:59:11 but Odl beryllium still causes the issue, and we are not sure we can fix that for Colorado 08:59:20 I have no objection, somehow it corresponds to the request from Frank for a scenario owner to be able to select the cases 08:59:22 would that be ok? 08:59:27 As said I have no objection 08:59:41 today there are not such mechanism, but it will not be difficult 08:59:45 for sure we need to add a comment in release notes 08:59:48 we should porobably do like in yardstick 08:59:53 ok 09:00:03 I will propose a patch then 09:00:19 craeate a os-odl_l2-bgpvpn-ha config file with just the list of test to be run 09:00:27 and it should overwritte the default mechanism 09:00:52 we should probably API it so scenrio owner could build this list and create automatically the list in the future 09:00:55 or just add a restriction in rally 09:01:10 ya, that's Danube :) 09:01:18 yep 09:01:37 since lxd there is already the possibility to black list cases in rally 09:01:40 ok, that was it, thanks 09:01:54 ok no objection from me, juhak? 09:02:06 for me the scenario owner is master on board... 09:02:09 fine for me 09:02:12 we might blacklist rally completelly for Colorado.1.0, and enable it again for Colorado.2.0 which comes with boron 09:02:26 ok 09:02:41 #info bgpvpn scenario blacklist rally completelly for Colorado.1.0, and enable it again for Colorado.2.0 which comes with boron 09:02:46 any other topic for AoB 09:02:53 not from me 09:03:16 just saw the mail from Chris, no more TSC chair next year...I need to organize also Functest election... 09:03:32 A comment has been added to a proposed change to functest: Add judgement after executing Sfc.py https://gerrit.opnfv.org/gerrit/20413 09:04:01 SerenaFeng: we did not talk about ELK, we will do it offline, I will contact the guy from bitergia 09:04:11 okey 09:04:17 no problem 09:04:19 #action morgan_orange contact bitergia / ELK 09:04:34 ok if no more question, it is all for this week 09:04:38 thanks for attending 09:04:43 enjoy the Colorado last miles 09:04:55 and let's start discussing of the Danube 09:05:01 #endmeeting